[2604.03592] Unveiling Language Routing Isolation in Multilingual MoE Models for Interpretable Subnetwork Adaptation

Computer Science > Computation and Language arXiv:2604.03592 (cs) [Submitted on 4 Apr 2026] Title:Unveiling Language Routing Isolation in Multilingual MoE Models for Interpretable Subnetwork Adaptation Authors:Kening Zheng, Wei-Chieh Huang, Jiahao Huo, Zhonghao Li, Henry Peng Zou, Yibo Yan, Xin Zou, Jungang Li, Junzhuo Li, Hanrong Zhang, Xuming Hu, Philip S. Yu View a PDF of the paper titled Unveiling Language Routing Isolation in Multilingual MoE Models for Interpretable Subnetwork Adaptation, by Kening Zheng and 11 other authors View PDF HTML (experimental) Abstract:Mixture-of-Experts (MoE) models exhibit striking performance disparities across languages, yet the internal mechanisms driving these gaps remain poorly understood. In this work, we conduct a systematic analysis of expert routing patterns in MoE models, revealing a phenomenon we term Language Routing Isolation, in which high- and low-resource languages tend to activate largely disjoint expert sets. Through layer-stratified analysis, we further show that routing patterns exhibit a layer-wise convergence-divergence pattern across model depth. Building on these findings, we propose RISE (Routing Isolation-guided Subnetwork Enhancement), a framework that exploits routing isolation to identify and adapt language-specific expert subnetworks. RISE applies a tripartite selection strategy, using specificity scores to identify language-specific experts in shallow and deep layers and overlap scores to select universal ex...