![[2602.16784] Omitted Variable Bias in Language Models Under Distribution Shift](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.16784] Omitted Variable Bias in Language Models Under Distribution Shift
Summary
This paper explores omitted variable bias in language models under distribution shifts, proposing a framework to evaluate and optimize performance in varying data conditions.
Why It Matters
Understanding omitted variable bias is crucial for improving the robustness of language models, especially as they are deployed in diverse real-world scenarios. This research provides insights into enhancing model evaluation and optimization techniques, which is vital for practitioners in machine learning and natural language processing.
Key Takeaways
- Omitted variable bias can significantly affect language model performance under distribution shifts.
- The paper introduces a framework to evaluate and optimize models based on unobserved variables.
- Empirical results demonstrate improved out-of-distribution performance using the proposed method.
Computer Science > Machine Learning arXiv:2602.16784 (cs) [Submitted on 18 Feb 2026] Title:Omitted Variable Bias in Language Models Under Distribution Shift Authors:Victoria Lin, Louis-Philippe Morency, Eli Ben-Michael View a PDF of the paper titled Omitted Variable Bias in Language Models Under Distribution Shift, by Victoria Lin and 2 other authors View PDF HTML (experimental) Abstract:Despite their impressive performance on a wide variety of tasks, modern language models remain susceptible to distribution shifts, exhibiting brittle behavior when evaluated on data that differs in distribution from their training data. In this paper, we describe how distribution shifts in language models can be separated into observable and unobservable components, and we discuss how established approaches for dealing with distribution shift address only the former. Importantly, we identify that the resulting omitted variable bias from unobserved variables can compromise both evaluation and optimization in language models. To address this challenge, we introduce a framework that maps the strength of the omitted variables to bounds on the worst-case generalization performance of language models under distribution shift. In empirical experiments, we show that using these bounds directly in language model evaluation and optimization provides more principled measures of out-of-distribution performance, improves true out-of-distribution performance relative to standard distribution shift adjus...
