[2604.03532] LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering

Computer Science > Computation and Language arXiv:2604.03532 (cs) [Submitted on 4 Apr 2026] Title:LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering Authors:Sing Hieng Wong, Hassan Sajjad, A.B. Siddique View a PDF of the paper titled LangFIR: Discovering Sparse Language-Specific Features from Monolingual Data for Language Steering, by Sing Hieng Wong and 2 other authors View PDF HTML (experimental) Abstract:Large language models (LLMs) show strong multilingual capabilities, yet reliably controlling the language of their outputs remains difficult. Representation-level steering addresses this by adding language-specific vectors to model activations at inference time, but identifying language-specific directions in the residual stream often relies on multilingual or parallel data that can be expensive to obtain. Sparse autoencoders (SAEs) decompose residual activations into interpretable, sparse feature directions and offer a natural basis for this search, yet existing SAE-based approaches face the same data constraint. We introduce LangFIR (Language Feature Identification via Random-token Filtering), a method that discovers language-specific SAE features using only a small amount of monolingual data and random-token sequences. Many SAE features consistently activated by target-language inputs do not encode language identity. Random-token sequences surface these language-agnostic features, allowing LangFIR to filter them out and...