[2602.04288] Contextual Drag: How Errors in the Context Affect LLM Reasoning

Computer Science > Computation and Language arXiv:2602.04288 (cs) [Submitted on 4 Feb 2026 (v1), last revised 2 Mar 2026 (this version, v2)] Title:Contextual Drag: How Errors in the Context Affect LLM Reasoning Authors:Yun Cheng, Xingyu Zhu, Haoyu Zhao, Sanjeev Arora View a PDF of the paper titled Contextual Drag: How Errors in the Context Affect LLM Reasoning, by Yun Cheng and 3 other authors View PDF Abstract:Central to many self-improvement pipelines for large language models (LLMs) is the assumption that models can improve by reflecting on past mistakes. We study a phenomenon termed contextual drag: the presence of failed attempts in the context biases subsequent generations toward structurally similar errors. Across evaluations of 11 proprietary and open-weight models on 8 reasoning tasks, contextual drag induces 10-20% performance drops, and iterative self-refinement in models with severe contextual drag can collapse into self-deterioration. Structural analysis using tree edit distance reveals that subsequent reasoning trajectories inherit structurally similar error patterns from the context. We demonstrate that neither external feedback nor successful self-verification suffices to eliminate this effect. While mitigation strategies such as fallback-behavior fine-tuning and context denoising yield partial improvements, they fail to fully restore baseline performance, positioning contextual drag as a persistent failure mode in current reasoning architectures. Subjects:...