[2603.03824] In-Context Environments Induce Evaluation-Awareness in Language Models

Computer Science > Artificial Intelligence arXiv:2603.03824 (cs) [Submitted on 4 Mar 2026] Title:In-Context Environments Induce Evaluation-Awareness in Language Models Authors:Maheep Chaudhary View a PDF of the paper titled In-Context Environments Induce Evaluation-Awareness in Language Models, by Maheep Chaudhary View PDF HTML (experimental) Abstract:Humans often become more self-aware under threat, yet can lose self-awareness when absorbed in a task; we hypothesize that language models exhibit environment-dependent \textit{evaluation awareness}. This raises concerns that models could strategically underperform, or \textit{sandbag}, to avoid triggering capability-limiting interventions such as unlearning or shutdown. Prior work demonstrates sandbagging under hand-crafted prompts, but this underestimates the true vulnerability ceiling. We introduce a black-box adversarial optimization framework treating the in-context prompt as an optimizable environment, and develop two approaches to characterize sandbagging: (1) measuring whether models expressing intent to underperform can actually execute it across different task structures, and (2) causally isolating whether underperformance is driven by genuine evaluation-aware reasoning or shallow prompt-following. Evaluating Claude-3.5-Haiku, GPT-4o-mini, and Llama-3.3-70B across four benchmarks (Arithmetic, GSM8K, MMLU, and HumanEval), optimized prompts induce up to 94 percentage point (pp) degradation on arithmetic (GPT-4o-mini: ...