![[2510.22037] ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2510.22037] ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality
Summary
The paper presents ATLAS, a study on adaptive transfer scaling laws for multilingual pretraining, finetuning, and decoding, based on extensive multilingual experiments.
Why It Matters
As AI models increasingly serve global audiences, understanding multilingual scaling laws is crucial for improving model performance across languages. This research provides insights that can help practitioners optimize AI systems for diverse linguistic contexts, addressing the limitations of English-centric models.
Key Takeaways
- Introduces the Adaptive Transfer Scaling Law (ATLAS) for multilingual AI models.
- Conducted 774 multilingual training experiments across 400+ languages.
- Identifies optimal strategies for scaling model size and data when adding languages.
- Provides empirical data on cross-lingual transfer benefits between language pairs.
- Offers guidance on when to pretrain from scratch versus finetuning existing models.
Computer Science > Computation and Language arXiv:2510.22037 (cs) [Submitted on 24 Oct 2025 (v1), last revised 25 Feb 2026 (this version, v2)] Title:ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality Authors:Shayne Longpre, Sneha Kudugunta, Niklas Muennighoff, I-Hung Hsu, Isaac Caswell, Alex Pentland, Sercan Arik, Chen-Yu Lee, Sayna Ebrahimi View a PDF of the paper titled ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality, by Shayne Longpre and 8 other authors View PDF HTML (experimental) Abstract:Scaling laws research has focused overwhelmingly on English -- yet the most prominent AI models explicitly serve billions of international users. In this work, we undertake the largest multilingual scaling laws study to date, totaling 774 multilingual training experiments, spanning 10M-8B model parameters, 400+ training languages and 48 evaluation languages. We introduce the Adaptive Transfer Scaling Law (ATLAS) for both monolingual and multilingual pretraining, which outperforms existing scaling laws' out-of-sample generalization often by more than 0.3 R^2. Our analyses of the experiments shed light on multilingual learning dynamics, transfer properties between languages, and the curse of multilinguality. First, we derive a cross-lingual transfer matrix, empirically measuring mutual benefit scores between 38 x 38=1444 language pairs. Secon...
