[2603.01291] JailNewsBench: Multi-Lingual and Regional Benchmark for Fake News Generation under Jailbreak Attacks

Computer Science > Machine Learning arXiv:2603.01291 (cs) [Submitted on 1 Mar 2026] Title:JailNewsBench: Multi-Lingual and Regional Benchmark for Fake News Generation under Jailbreak Attacks Authors:Masahiro Kaneko, Ayana Niwa, Timothy Baldwin View a PDF of the paper titled JailNewsBench: Multi-Lingual and Regional Benchmark for Fake News Generation under Jailbreak Attacks, by Masahiro Kaneko and 2 other authors View PDF Abstract:Fake news undermines societal trust and decision-making across politics, economics, health, and international relations, and in extreme cases threatens human lives and societal safety. Because fake news reflects region-specific political, social, and cultural contexts and is expressed in language, evaluating the risks of large language models (LLMs) requires a multi-lingual and regional perspective. Malicious users can bypass safeguards through jailbreak attacks, inducing LLMs to generate fake news. However, no benchmark currently exists to systematically assess attack resilience across languages and regions. Here, we propose JailNewsBench, the first benchmark for evaluating LLM robustness against jailbreak-induced fake news generation. JailNewsBench spans 34 regions and 22 languages, covering 8 evaluation sub-metrics through LLM-as-a-Judge and 5 jailbreak attacks, with approximately 300k instances. Our evaluation of 9 LLMs reveals that the maximum attack success rate (ASR) reached 86.3% and the maximum harmfulness score was 3.5 out of 5. Notably,...