[2505.21505] How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspective

Computer Science > Computation and Language arXiv:2505.21505 (cs) [Submitted on 27 May 2025 (v1), last revised 1 Apr 2026 (this version, v3)] Title:How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspective Authors:Shimao Zhang, Zhejian Lai, Xiang Liu, Shuaijie She, Xiao Liu, Yeyun Gong, Shujian Huang, Jiajun Chen View a PDF of the paper titled How Does Alignment Enhance LLMs' Multilingual Capabilities? A Language Neurons Perspective, by Shimao Zhang and 7 other authors View PDF HTML (experimental) Abstract:Multilingual Alignment is an effective and representative paradigm to enhance LLMs' multilingual capabilities, which transfers the capabilities from the high-resource languages to the low-resource languages. Meanwhile, some research on language-specific neurons provides a new perspective to analyze and understand LLMs' mechanisms. However, we find that there are many neurons that are shared by multiple but not all languages and cannot be correctly classified. In this work, we propose a ternary classification methodology that categorizes neurons into three types, including language-specific neurons, language-related neurons, and general neurons. And we propose a corresponding identification algorithm to distinguish these different types of neurons. Furthermore, based on the distributional characteristics of different types of neurons, we divide the LLMs' internal process for multilingual inference into four parts: (1) multilingual understa...