![[2507.05411] AXLearn: Modular, Hardware-Agnostic Large Model Training](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2507.05411] AXLearn: Modular, Hardware-Agnostic Large Model Training
Summary
AXLearn presents a modular and hardware-agnostic approach to training large deep learning models, enhancing scalability and performance while simplifying integration.
Why It Matters
With the growing demand for efficient large model training, AXLearn addresses critical challenges in scalability and hardware compatibility. Its modular design allows for rapid experimentation and integration of new features, making it a significant advancement in deep learning infrastructure.
Key Takeaways
- AXLearn enables scalable training of large models with consistent performance.
- The system's modular architecture allows for easy integration of new features.
- AXLearn maintains constant complexity, improving efficiency compared to traditional systems.
Computer Science > Machine Learning arXiv:2507.05411 (cs) [Submitted on 7 Jul 2025 (v1), last revised 19 Feb 2026 (this version, v3)] Title:AXLearn: Modular, Hardware-Agnostic Large Model Training Authors:Mark Lee, Chang Lan, Tom Gunter, John Peebles, Hanzhi Zhou, Kelvin Zou, Sneha Bangalore, Chung-Cheng Chiu, Nan Du, Xianzhi Du, Philipp Dufter, Ruixuan Hou, Haoshuo Huang, Dongseong Hwang, Xiang Kong, Jinhao Lei, Tao Lei, Meng Li, Li Li, Jiarui Lu, Zhiyun Lu, Yiping Ma, David Qiu, Vivek Rathod, Senyu Tong, Zhucheng Tu, Jianyu Wang, Yongqiang Wang, Zirui Wang, Floris Weers, Sam Wiseman, Guoli Yin, Bowen Zhang, Xiyou Zhou, Danyang Zhuo, Cheng Leong, Ruoming Pang View a PDF of the paper titled AXLearn: Modular, Hardware-Agnostic Large Model Training, by Mark Lee and 36 other authors View PDF HTML (experimental) Abstract:AXLearn is a production system which facilitates scalable and high-performance training of large deep learning models. Compared to other state-of-art deep learning systems, AXLearn has a unique focus on modularity and support for hardware-agnostic training. AXLearn's internal interfaces between software components follow strict encapsulation, allowing different components to be assembled to facilitate rapid model development and experimentation on different hardware infrastructure. AXLearn maintains constant complexity as we scale the components in the system, compared to linear or quadratic complexity in state-of-the-art training systems. This allows integrat...