[2603.28651] Not Search, But Scan: Benchmarking MLLMs on Scan-Oriented Academic Paper Reasoning

Computer Science > Artificial Intelligence arXiv:2603.28651 (cs) [Submitted on 27 Mar 2026] Title:Not Search, But Scan: Benchmarking MLLMs on Scan-Oriented Academic Paper Reasoning Authors:Rongjin Li, Zichen Tang, Xianghe Wang, Xinyi Hu, Zhengyu Wang, Zhengyu Lu, Yiling Huang, Jiayuan Chen, Weisheng Tan, Jiacheng Liu, Zhongjun Yang, Haihong E View a PDF of the paper titled Not Search, But Scan: Benchmarking MLLMs on Scan-Oriented Academic Paper Reasoning, by Rongjin Li and 11 other authors View PDF HTML (experimental) Abstract:With the rapid progress of multimodal large language models (MLLMs), AI already performs well at literature retrieval and certain reasoning tasks, serving as a capable assistant to human researchers, yet it remains far from autonomous research. The fundamental reason is that current work on academic paper reasoning is largely confined to a search-oriented paradigm centered on pre-specified targets, with reasoning grounded in relevance retrieval, which struggles to support researcher-style full-document understanding, reasoning, and verification. To bridge this gap, we propose \textbf{ScholScan}, a new benchmark for academic paper reasoning. ScholScan introduces a scan-oriented task setting that asks models to read and cross-check entire papers like human researchers, scanning the document to identify consistency issues. The benchmark comprises 1,800 carefully annotated questions drawn from nine error categories across 13 natural-science domains and 7...