[2510.13849] Language steering in latent space to mitigate unintended code-switching

Computer Science > Computation and Language arXiv:2510.13849 (cs) [Submitted on 11 Oct 2025 (v1), last revised 2 Mar 2026 (this version, v2)] Title:Language steering in latent space to mitigate unintended code-switching Authors:Andrey Goncharov, Nikolai Kondusov, Alexey Zaytsev View a PDF of the paper titled Language steering in latent space to mitigate unintended code-switching, by Andrey Goncharov and 2 other authors View PDF HTML (experimental) Abstract:Multilingual Large Language Models (LLMs) often exhibit unintended code-switching, reducing reliability in downstream tasks. We propose latent-space language steering, a lightweight inference-time method that identifies language directions via PCA on parallel translations and steers token embeddings along these axes to control language identity. Our approach mitigates code-switching while preserving semantics with negligible computational overhead and requires only minimal parallel data for calibration. Empirically, we achieve 95-99\% language classification accuracy using a single principal component and reduce next-token distributional divergence by up to 55\% across multiple language pairs on Qwen2.5 and Llama-3.2 models. Generation-based evaluation on Llama-3.2 further demonstrates 63--99\% reduction in Code-Switching Index across four language pairs ($p < 0.001$). We further analyze the layer-wise evolution of language representations, revealing that language identity concentrates in final layers with near-perfect l...