[2603.30036] Aligned, Orthogonal or In-conflict: When can we safely optimize Chain-of-Thought?

Computer Science > Machine Learning arXiv:2603.30036 (cs) [Submitted on 31 Mar 2026] Title:Aligned, Orthogonal or In-conflict: When can we safely optimize Chain-of-Thought? Authors:Max Kaufmann, David Lindner, Roland S. Zimmermann, and Rohin Shah View a PDF of the paper titled Aligned, Orthogonal or In-conflict: When can we safely optimize Chain-of-Thought?, by Max Kaufmann and David Lindner and Roland S. Zimmermann and and Rohin Shah View PDF HTML (experimental) Abstract:Chain-of-Thought (CoT) monitoring, in which automated systems monitor the CoT of an LLM, is a promising approach for effectively overseeing AI systems. However, the extent to which a model's CoT helps us oversee the model - the monitorability of the CoT - can be affected by training, for instance by the model learning to hide important features of its reasoning. We propose and empirically validate a conceptual framework for predicting when and why this occurs. We model LLM post-training as an RL environment where the reward decomposes into two terms: one term depending on final outputs and another term depending on the CoT. Our framework allows us to classify these two terms as "aligned", "orthogonal", or "in-conflict" before training. We predict that training with in-conflict terms will reduce monitorability, orthogonal terms will not affect it, and aligned terms will improve it. To validate our framework, we use it to classify a set of RL environments, train LLMs within those environments, and evaluate ...