[2603.03308] Old Habits Die Hard: How Conversational History Geometrically Traps LLMs

Computer Science > Computation and Language arXiv:2603.03308 (cs) [Submitted on 8 Feb 2026] Title:Old Habits Die Hard: How Conversational History Geometrically Traps LLMs Authors:Adi Simhi, Fazl Barez, Martin Tutek, Yonatan Belinkov, Shay B. Cohen View a PDF of the paper titled Old Habits Die Hard: How Conversational History Geometrically Traps LLMs, by Adi Simhi and 4 other authors View PDF HTML (experimental) Abstract:How does the conversational past of large language models (LLMs) influence their future performance? Recent work suggests that LLMs are affected by their conversational history in unexpected ways. For instance, hallucinations in prior interactions may influence subsequent model responses. In this work, we introduce History-Echoes, a framework that investigates how conversational history biases subsequent generations. The framework explores this bias from two perspectives: probabilistically, we model conversations as Markov chains to quantify state consistency; geometrically, we measure the consistency of consecutive hidden representations. Across three model families and six datasets spanning diverse phenomena, our analysis reveals a strong correlation between the two perspectives. By bridging these perspectives, we demonstrate that behavioral persistence manifests as a geometric trap, where gaps in the latent space confine the model's trajectory. Code available at this https URL. Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI) A...