![[2602.21372] The Mean is the Mirage: Entropy-Adaptive Model Merging under Heterogeneous Domain Shifts in Medical Imaging](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.21372] The Mean is the Mirage: Entropy-Adaptive Model Merging under Heterogeneous Domain Shifts in Medical Imaging
Summary
This article presents an entropy-adaptive model merging technique for medical imaging that addresses challenges posed by heterogeneous domain shifts, improving model performance without requiring labeled data.
Why It Matters
As medical imaging increasingly relies on machine learning, the ability to adapt models to unseen clinical environments is crucial. This research offers a solution to improve model reliability and accuracy, which is vital for patient care and diagnostic processes.
Key Takeaways
- Mean averaging in model merging can lead to unreliable outcomes under domain shifts.
- The proposed method adapts models in real-time using only forward passes, enhancing efficiency.
- Decoupling encoders and classifiers mitigates performance issues associated with model merging.
- Extensive evaluations show consistent performance improvements across various datasets.
- The method retains single-model inference, simplifying deployment in clinical settings.
Computer Science > Machine Learning arXiv:2602.21372 (cs) [Submitted on 24 Feb 2026] Title:The Mean is the Mirage: Entropy-Adaptive Model Merging under Heterogeneous Domain Shifts in Medical Imaging Authors:Sameer Ambekar, Reza Nasirigerdeh, Peter J. Schuffler, Lina Felsner, Daniel M. Lang, Julia A. Schnabel View a PDF of the paper titled The Mean is the Mirage: Entropy-Adaptive Model Merging under Heterogeneous Domain Shifts in Medical Imaging, by Sameer Ambekar and 5 other authors View PDF HTML (experimental) Abstract:Model merging under unseen test-time distribution shifts often renders naive strategies, such as mean averaging unreliable. This challenge is especially acute in medical imaging, where models are fine-tuned locally at clinics on private data, producing domain-specific models that differ by scanner, protocol, and population. When deployed at an unseen clinical site, test cases arrive in unlabeled, non-i.i.d. batches, and the model must adapt immediately without labels. In this work, we introduce an entropy-adaptive, fully online model-merging method that yields a batch-specific merged model via only forward passes, effectively leveraging target information. We further demonstrate why mean merging is prone to failure and misaligned under heterogeneous domain shifts. Next, we mitigate encoder classifier mismatch by decoupling the encoder and classification head, merging with separate merging coefficients. We extensively evaluate our method with state-of-the-ar...
