[2603.19275] Improving Automatic Summarization of Radiology Reports through Mid-Training of Large Language Models

Computer Science > Computation and Language arXiv:2603.19275 (cs) [Submitted on 28 Feb 2026] Title:Improving Automatic Summarization of Radiology Reports through Mid-Training of Large Language Models Authors:Mengxian Lyu, Cheng Peng, Ziyi Chen, Mengyuan Zhang, Jieting Li Lu, Yonghui Wu View a PDF of the paper titled Improving Automatic Summarization of Radiology Reports through Mid-Training of Large Language Models, by Mengxian Lyu and 5 other authors View PDF Abstract:Automatic summarization of radiology reports is an essential application to reduce the burden on physicians. Previous studies have widely used the "pre-training, fine-tuning" strategy to adapt large language models (LLMs) for summarization. This study proposed a subdomain adaptation through a mid-training method to improve summarization. We explored three adaptation strategies: (1) general-domain pre-training, (2) clinical-domain pre-training, and (3) clinical-domain pre-training followed by subdomain mid-training. We developed models using large-scale clinical text from the University of Florida (UF) Health and conducted mid-training and fine-tuning experiments using widely used benchmark datasets including OpenI and MIMIC-CXR. The experimental results show that the mid-trained model, GatorTronT5-Radio, achieved the best performance, outperforming models without mid-training in both text-based measures (ROUGE-L) and factuality measures (RadGraph-F1). Our mid-training methods also demonstrate better few-shot...