[2603.28407] MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome

Computer Science > Artificial Intelligence arXiv:2603.28407 (cs) [Submitted on 30 Mar 2026] Title:MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome Authors:Fangda Ye, Yuxin Hu, Pengxiang Zhu, Yibo Li, Ziqi Jin, Yao Xiao, Yibo Wang, Lei Wang, Zhen Zhang, Lu Wang, Yue Deng, Bin Wang, Yifan Zhang, Liangcai Su, Xinyu Wang, He Zhao, Chen Wei, Qiang Ren, Bryan Hooi, An Bo, Shuicheng Yan, Lidong Bing View a PDF of the paper titled MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome, by Fangda Ye and 21 other authors View PDF HTML (experimental) Abstract:Recent progress in deep research systems has been impressive, but evaluation still lags behind real user needs. Existing benchmarks predominantly assess final reports using fixed rubrics, failing to evaluate the underlying research process. Most also offer limited multimodal coverage, rely on synthetic tasks that do not reflect real-world query complexity, and cannot be refreshed as knowledge evolves. To address these gaps, we introduce MiroEval, a benchmark and evaluation framework for deep research systems. The benchmark comprises 100 tasks (70 text-only, 30 multimodal), all grounded in real user needs and constructed via a dual-path pipeline that supports periodic updates, enabling a live and evolving setting. The proposed evaluation suite assesses deep research systems along three complementary dimensions: adaptive synthesis quality evaluation with task-specific rubrics, ag...