[2603.04969] MPCEval: A Benchmark for Multi-Party Conversation Generation

Computer Science > Computation and Language arXiv:2603.04969 (cs) [Submitted on 5 Mar 2026] Title:MPCEval: A Benchmark for Multi-Party Conversation Generation Authors:Minxing Zhang, Yi Yang, Zhuofan Jia, Xuan Yang, Jian Pei, Yuchen Zang, Xingwang Deng, Xianglong Chen View a PDF of the paper titled MPCEval: A Benchmark for Multi-Party Conversation Generation, by Minxing Zhang and 7 other authors View PDF HTML (experimental) Abstract:Multi-party conversation generation, such as smart reply and collaborative assistants, is an increasingly important capability of generative AI, yet its evaluation remains a critical bottleneck. Compared to two-party dialogue, multi-party settings introduce distinct challenges, including complex turn-taking, role-dependent speaker behavior, long-range conversational structure, and multiple equally valid continuations. Accordingly, we introduce MPCEval, a task-aware evaluation and benchmarking suite for multi-party conversation generation. MPCEval decomposes generation quality into speaker modeling, content quality, and speaker--content consistency, and explicitly distinguishes local next-turn prediction from global full-conversation generation. It provides novel, quantitative, reference-free, and reproducible metrics that scale across datasets and models. We apply MPCEval to diverse public and real-world datasets and evaluate modern generation methods alongside human-authored conversations. The results reveal systematic, dimension-specific model...