![[2506.05850] Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Models](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2506.05850] Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Models
Summary
The paper investigates 'Cross-lingual Collapse' in large language models (LLMs), revealing how reasoning capabilities can revert to a dominant pre-training language, impacting multilingual performance.
Why It Matters
Understanding Cross-lingual Collapse is crucial for improving multilingual LLMs. It highlights the trade-off between reasoning depth and language fidelity, which is essential for developing more effective AI systems that can perform well across different languages.
Key Takeaways
- Cross-lingual Collapse occurs when multilingual LLMs revert to their dominant pre-training language during reasoning.
- The phenomenon is amplified by English-centric priors and long-CoT optimization.
- Interventions can mitigate collapse but reveal a trade-off between performance and language fidelity.
Computer Science > Computation and Language arXiv:2506.05850 (cs) [Submitted on 6 Jun 2025 (v1), last revised 23 Feb 2026 (this version, v3)] Title:Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Models Authors:Cheonbok Park, Jeonghoon Kim, Joosung Lee, Sanghwan Bae, Jaegul Choo, Kang Min Yoo View a PDF of the paper titled Cross-lingual Collapse: How Language-Centric Foundation Models Shape Reasoning in Large Language Models, by Cheonbok Park and 5 other authors View PDF HTML (experimental) Abstract:Reinforcement learning with verifiable reward (RLVR) has been instrumental in eliciting strong reasoning capabilities from large language models (LLMs) via long chains of thought (CoT). During RLVR training, we formalize and systemically study an empirical phenomenon whereby a multilingual model's CoT reverts to its dominant pre-training language (e.g., English) even when prompted in another language, which we term Cross-lingual Collapse. Because the long-CoT regime magnifies exposure to linguistic priors, the underlying trade-off between maximizing reasoning depth and preserving target-language fidelity has remained under-characterized. To examine this trade-off, we train LLMs with Group-Relative Policy Optimization (GRPO) on translated versions of math datasets widely used to elicit long-CoT reasoning. Throughout training, we track both task accuracy and the language consistency of reasoning chains. Our experiments yield three ...
