[2503.08292] Large Language Models for Outpatient Referral: Problem Definition, Benchmarking and Challenges

Computer Science > Computation and Language arXiv:2503.08292 (cs) [Submitted on 11 Mar 2025 (v1), last revised 8 Apr 2026 (this version, v4)] Title:Large Language Models for Outpatient Referral: Problem Definition, Benchmarking and Challenges Authors:Xiaoxiao Liu, Qingying Xiao, Bingquan Zhang, Junying Chen, Xiangyi Feng, Ziniu Li, Xiang Wan, Jian Chang, Guangjun Yu, Yan Hu, Benyou Wang View a PDF of the paper titled Large Language Models for Outpatient Referral: Problem Definition, Benchmarking and Challenges, by Xiaoxiao Liu and 10 other authors View PDF HTML (experimental) Abstract:Large language models (LLMs) are increasingly applied to outpatient referral tasks across healthcare systems. However, there is a lack of standardized evaluation criteria to assess their effectiveness, particularly in dynamic, interactive scenarios. In this study, we systematically examine the capabilities and limitations of LLMs in managing tasks within Intelligent Outpatient Referral (IOR) systems and propose a comprehensive evaluation framework specifically designed for such systems. This framework comprises two core tasks: static evaluation, which focuses on evaluating the ability of predefined outpatient referrals, and dynamic evaluation, which evaluates capabilities of refining outpatient referral recommendations through iterative dialogues. Our findings suggest that LLMs offer limited advantages over BERT-like models, but show promise in asking effective questions during interactive di...