[2603.21376] A transformer architecture alteration to incentivise externalised reasoning

Computer Science > Artificial Intelligence arXiv:2603.21376 (cs) [Submitted on 22 Mar 2026] Title:A transformer architecture alteration to incentivise externalised reasoning Authors:Elizabeth Pavlova, Mariia Koroliuk, Karthik Viswanathan, Cameron Tice, Edward James Young, Puria Radmard View a PDF of the paper titled A transformer architecture alteration to incentivise externalised reasoning, by Elizabeth Pavlova and 5 other authors View PDF HTML (experimental) Abstract:We propose a new architectural change, and post-training pipeline, for making LLMs more verbose reasoners by teaching a model to truncate forward passes early. We augment an existing transformer architecture with an early-exit mechanism at intermediate layers and train the model to exit at shallower layers when the next token can be predicted without deep computation. After a calibration stage, we incentivise the model to exit as early as possible while maintaining task performance using reinforcement learning. We provide preliminary results to this effect for small reasoning models, showing that they learn to adaptively reduce computations across tokens. We predict that, applied at the right scale, our approach can minimise the amount of excess computation that reasoning models have at their disposal to perform non-myopic planning using their internal activations, reserving this only for difficult-to-predict tokens. Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2603.21376 [cs.AI]   (or arXiv:2603...