![[2505.19558] PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2505.19558] PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives
Summary
The paper introduces PoliCon, a benchmark for evaluating large language models (LLMs) in achieving political consensus from diverse party perspectives using European Parliament deliberation records.
Why It Matters
PoliCon addresses the gap in understanding LLMs' capabilities in political contexts, crucial for enhancing governance and decision-making. By evaluating LLMs on their ability to draft consensus resolutions, it provides insights into their biases and effectiveness in real-world political scenarios.
Key Takeaways
- PoliCon is based on 2,225 deliberation records from the European Parliament.
- The benchmark evaluates LLMs on their ability to draft resolutions considering political diversity.
- Results indicate that current LLMs struggle with complex consensus tasks and exhibit partisan biases.
- The study highlights the need for improved LLM capabilities in political consensus-building.
- PoliCon's framework can aid future research in AI's role in governance.
Computer Science > Computers and Society arXiv:2505.19558 (cs) [Submitted on 26 May 2025 (v1), last revised 13 Feb 2026 (this version, v3)] Title:PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives Authors:Zhaowei Zhang, Xiaobo Wang, Minghua Yi, Mengmeng Wang, Fengshuo Bai, Zilong Zheng, Yipeng Kang, Yaodong Yang View a PDF of the paper titled PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives, by Zhaowei Zhang and 7 other authors View PDF HTML (experimental) Abstract:Achieving political consensus is crucial yet challenging for the effective functioning of social governance. However, although frontier AI systems represented by large language models (LLMs) have developed rapidly in recent years, their capabilities in this scope are still understudied. In this paper, we introduce PoliCon, a novel benchmark constructed from 2,225 high-quality deliberation records of the European Parliament over 13 years, ranging from 2009 to 2022, to evaluate the ability of LLMs to draft consensus resolutions based on divergent party positions under varying collective decision-making contexts and political requirements. Specifically, PoliCon incorporates four factors to build each task environment for finding different political consensus: specific political issues, political goals, participating parties, and power structures based on seat distribution. We also developed an evaluation framework based on social choice theory for PoliCon, which...
