[2603.21329] COINBench: Moving Beyond Individual Perspectives to Collective Intent Understanding

Computer Science > Information Retrieval arXiv:2603.21329 (cs) [Submitted on 22 Mar 2026] Title:COINBench: Moving Beyond Individual Perspectives to Collective Intent Understanding Authors:Xiaozhe Li, Tianyi Lyu, Siyi Yang, Yizhao Yang, Yuxi Gong, Jinxuan Huang, Ligao Zhang, Zhuoyi Huang, Qingwen Liu View a PDF of the paper titled COINBench: Moving Beyond Individual Perspectives to Collective Intent Understanding, by Xiaozhe Li and 8 other authors View PDF HTML (experimental) Abstract:Understanding human intent is a high-level cognitive challenge for Large Language Models (LLMs), requiring sophisticated reasoning over noisy, conflicting, and non-linear discourse. While LLMs excel at following individual instructions, their ability to distill Collective Intent - the process of extracting consensus, resolving contradictions, and inferring latent trends from multi-source public discussions - remains largely unexplored. To bridge this gap, we introduce COIN-BENCH, a dynamic, real-world, live-updating benchmark specifically designed to evaluate LLMs on collective intent understanding within the consumer domain. Unlike traditional benchmarks that focus on transactional outcomes, COIN-BENCH operationalizes intent as a hierarchical cognitive structure, ranging from explicit scenarios to deep causal reasoning. We implement a robust evaluation pipeline that combines a rule-based method with an LLM-as-the-Judge approach. This framework incorporates COIN-TREE for hierarchical cognitive...