![[2602.18658] Communication-Efficient Personalized Adaptation via Federated-Local Model Merging](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.18658] Communication-Efficient Personalized Adaptation via Federated-Local Model Merging
Summary
The paper presents Potara, a framework for federated personalization that merges general and personalized models, improving efficiency and performance in machine learning tasks.
Why It Matters
As machine learning models become increasingly complex, efficient personalization in federated settings is crucial. Potara addresses the challenges of task-level heterogeneity and communication costs, offering a theoretically grounded solution that enhances model performance while minimizing resource usage. This work is significant for researchers and practitioners in AI looking to optimize federated learning systems.
Key Takeaways
- Potara merges federated and local models for personalized adaptation.
- The framework minimizes communication costs while enhancing performance.
- Closed-form optimal mixing weights are derived for improved model efficiency.
- Experiments demonstrate Potara's effectiveness on vision and language benchmarks.
- The approach provides a theoretical basis for model merging in federated learning.
Computer Science > Machine Learning arXiv:2602.18658 (cs) [Submitted on 20 Feb 2026] Title:Communication-Efficient Personalized Adaptation via Federated-Local Model Merging Authors:Yinan Zou, Md Kamran Chowdhury Shisher, Christopher G. Brinton, Vishrant Tripathi View a PDF of the paper titled Communication-Efficient Personalized Adaptation via Federated-Local Model Merging, by Yinan Zou and 3 other authors View PDF HTML (experimental) Abstract:Parameter-efficient fine-tuning methods, such as LoRA, offer a practical way to adapt large vision and language models to client tasks. However, this becomes particularly challenging under task-level heterogeneity in federated deployments. In this regime, personalization requires balancing general knowledge with personalized knowledge, yet existing approaches largely rely on heuristic mixing rules and lack theoretical justification. Moreover, prior model merging approaches are also computation and communication intensive, making the process inefficient in federated settings. In this work, we propose Potara, a principled framework for federated personalization that constructs a personalized model for each client by merging two complementary models: (i) a federated model capturing general knowledge, and (ii) a local model capturing personalized knowledge. Through the construct of linear mode connectivity, we show that the expected task loss admits a variance trace upper bound, whose minimization yields closed-form optimal mixing weight...
