[2603.21460] When Documents Disagree: Measuring Institutional Variation in Transplant Guidance with Retrieval-Augmented Language Models

Computer Science > Information Retrieval arXiv:2603.21460 (cs) [Submitted on 23 Mar 2026] Title:When Documents Disagree: Measuring Institutional Variation in Transplant Guidance with Retrieval-Augmented Language Models Authors:Yubo Li, Ramayya Krishnan, Rema Padman View a PDF of the paper titled When Documents Disagree: Measuring Institutional Variation in Transplant Guidance with Retrieval-Augmented Language Models, by Yubo Li and 2 other authors View PDF HTML (experimental) Abstract:Patient education materials for solid-organ transplantation vary substantially across U.S. centers, yet no systematic method exists to quantify this heterogeneity at scale. We introduce a framework that grounds the same patient questions in different centers' handbooks using retrieval-augmented language models and compares the resulting answers using a five-label consistency taxonomy. Applied to 102 handbooks from 23 centers and 1,115 benchmark questions, the framework quantifies heterogeneity across four dimensions: question, topic, organ, and center. We find that 20.8% of non-absent pairwise comparisons exhibit clinically meaningful divergence, concentrated in condition monitoring and lifestyle topics. Coverage gaps are even more prominent: 96.2% of question-handbook pairs miss relevant content, with reproductive health at 95.1% absence. Center-level divergence profiles are stable and interpretable, where heterogeneity reflects systematic institutional differences, likely due to patient div...