[2602.24277] Resources for Automated Evaluation of Assistive RAG Systems that Help Readers with News Trustworthiness Assessment

Computer Science > Information Retrieval arXiv:2602.24277 (cs) [Submitted on 27 Feb 2026] Title:Resources for Automated Evaluation of Assistive RAG Systems that Help Readers with News Trustworthiness Assessment Authors:Dake Zhang, Mark D. Smucker, Charles L. A. Clarke View a PDF of the paper titled Resources for Automated Evaluation of Assistive RAG Systems that Help Readers with News Trustworthiness Assessment, by Dake Zhang and 2 other authors View PDF HTML (experimental) Abstract:Many readers today struggle to assess the trustworthiness of online news because reliable reporting coexists with misinformation. The TREC 2025 DRAGUN (Detection, Retrieval, and Augmented Generation for Understanding News) Track provided a venue for researchers to develop and evaluate assistive RAG systems that support readers' news trustworthiness assessment by producing reader-oriented, well-attributed reports. As the organizers of the DRAGUN track, we describe the resources that we have newly developed to allow for the reuse of the track's tasks. The track had two tasks: (Task 1) Question Generation, producing 10 ranked investigative questions; and (Task 2, the main task) Report Generation, producing a 250-word report grounded in the MS MARCO V2.1 Segmented Corpus. As part of the track's evaluation, we had TREC assessors create importance-weighted rubrics of questions with expected short answers for 30 different news articles. These rubrics represent the information that assessors believe is...