![[2602.13273] MergePipe: A Budget-Aware Parameter Management System for Scalable LLM Merging](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.13273] MergePipe: A Budget-Aware Parameter Management System for Scalable LLM Merging
Summary
MergePipe is a novel parameter management system designed to enhance the scalability of large language model (LLM) merging by optimizing I/O operations and execution efficiency.
Why It Matters
As LLMs become increasingly integral to AI applications, efficient merging of these models is crucial for performance and resource management. MergePipe addresses significant inefficiencies in current methods, offering a scalable solution that could impact various AI development pipelines.
Key Takeaways
- MergePipe introduces a catalog-driven approach to LLM merging, improving parameter management.
- The system employs a cost-aware planner to optimize I/O operations, significantly reducing merge costs.
- Experiments demonstrate up to 11x speed improvements and a reduction in total I/O by an order of magnitude.
Computer Science > Databases arXiv:2602.13273 (cs) [Submitted on 5 Feb 2026] Title:MergePipe: A Budget-Aware Parameter Management System for Scalable LLM Merging Authors:Yuanyi Wang, Yanggan Gu, Zihao Wang, Kunxi Li, Yifan Yang, Zhaoyi Yan, Congkai Xie, Jianmin Wu, Hongxia Yang View a PDF of the paper titled MergePipe: A Budget-Aware Parameter Management System for Scalable LLM Merging, by Yuanyi Wang and 8 other authors View PDF HTML (experimental) Abstract:Large language model (LLM) merging has become a key technique in modern LLM development pipelines, enabling the integration of multiple task- or domain-specific expert models without retraining. However, as the number of experts grows, existing merging implementations treat model parameters as unstructured files and execute merges in a stateless, one-shot manner, leading to excessive disk I/O, redundant parameter scans, and poor scalability. In this paper, we present \textbf{MergePipe}, a parameter management system for scalable LLM merging. MergePipe is the first system that treats LLM merging as a data management and execution problem, and introduces a catalog-driven abstraction over model parameters, merge plans, and execution lineage. At its core, MergePipe employs a cost-aware planner that explicitly models expert parameter I/O and enforces user-specified I/O budgets, followed by a streaming execution engine that materializes merged models under transactional guarantees. Our key insight is that while base model re...