[2603.03541] RAG-X: Systematic Diagnosis of Retrieval-Augmented Generation for Medical Question Answering

Computer Science > Computation and Language arXiv:2603.03541 (cs) [Submitted on 3 Mar 2026] Title:RAG-X: Systematic Diagnosis of Retrieval-Augmented Generation for Medical Question Answering Authors:Aswini Sivakumar, Vijayan Sugumaran, Yao Qiang View a PDF of the paper titled RAG-X: Systematic Diagnosis of Retrieval-Augmented Generation for Medical Question Answering, by Aswini Sivakumar and 2 other authors View PDF HTML (experimental) Abstract:Automated question-answering (QA) systems increasingly rely on retrieval-augmented generation (RAG) to ground large language models (LLMs) in authoritative medical knowledge, ensuring clinical accuracy and patient safety in Artificial Intelligence (AI) applications for healthcare. Despite progress in RAG evaluation, current benchmarks focus only on simple multiple-choice QA tasks and employ metrics that poorly capture the semantic precision required for complex QA tasks. These approaches fail to diagnose whether an error stems from faulty retrieval or flawed generation, limiting developers from performing targeted improvement. To address this gap, we propose RAG-X, a diagnostic framework that evaluates the retriever and generator independently across a triad of QA tasks: information extraction, short-answer generation, and multiple-choice question (MCQ) answering. RAG-X introduces Context Utilization Efficiency (CUE) metrics to disaggregate system success into interpretable quadrants, isolating verified grounding from deceptive accu...