![[2602.12526] Constraint-Rectified Training for Efficient Chain-of-Thought](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.12526] Constraint-Rectified Training for Efficient Chain-of-Thought
Summary
The paper presents Constraint-Rectified Training (CRT), a framework designed to enhance the efficiency of Chain-of-Thought reasoning in Large Language Models by balancing reasoning length and accuracy.
Why It Matters
As Large Language Models become integral in various applications, optimizing their reasoning capabilities is crucial. CRT addresses the trade-off between reasoning length and accuracy, potentially leading to more efficient AI systems that can perform complex tasks without excessive computational costs.
Key Takeaways
- CRT minimizes reasoning length while maintaining accuracy in LLMs.
- The framework uses reference-guarded constrained optimization for stable performance.
- CRT reduces token usage and internal redundancy in responses.
- A two-stage training scheme helps discover optimal reasoning patterns.
- Intermediate checkpoints allow for control over reasoning verbosity.
Computer Science > Machine Learning arXiv:2602.12526 (cs) [Submitted on 13 Feb 2026] Title:Constraint-Rectified Training for Efficient Chain-of-Thought Authors:Qinhang Wu, Sen Lin, Ming Zhang, Yingbin Liang, Ness B. Shroff View a PDF of the paper titled Constraint-Rectified Training for Efficient Chain-of-Thought, by Qinhang Wu and 4 other authors View PDF HTML (experimental) Abstract:Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs), especially when combined with reinforcement learning (RL) based post-training methods. While longer reasoning traces can improve answer quality and unlock abilities such as self-correction, they also incur high inference costs and often introduce redundant steps, known as overthinking. Recent research seeks to develop efficient reasoning strategies that balance reasoning length and accuracy, either through length-aware reward design or prompt-based calibration. However, these heuristic-based approaches may suffer from severe accuracy drop and be very sensitive to hyperparameters. To address these problems, we introduce CRT (Constraint-Rectified Training), a principled post-training framework based on reference-guarded constrained optimization, yielding a more stable and interpretable formulation for efficient reasoning. CRT alternates between minimizing reasoning length and rectifying accuracy only when performance falls below the reference, enabling stable and effective pruning of re...
