[2603.01683] Surgical Post-Training: Cutting Errors, Keeping Knowledge

Computer Science > Computation and Language arXiv:2603.01683 (cs) [Submitted on 2 Mar 2026] Title:Surgical Post-Training: Cutting Errors, Keeping Knowledge Authors:Wenye Lin, Kai Han View a PDF of the paper titled Surgical Post-Training: Cutting Errors, Keeping Knowledge, by Wenye Lin and 1 other authors View PDF HTML (experimental) Abstract:Enhancing the reasoning capabilities of Large Language Models (LLMs) via post-training is often constrained by the trade-off between efficiency and catastrophic forgetting. While prior research emphasizes the role of on-policy data in mitigating forgetting, we uncover--and validate both theoretically and empirically--an overlooked yet critical mechanism: the implicit regularization inherent in Direct Preference Optimization's (DPO) reward estimate. This motivates our Surgical Post-Training (SPoT), a new paradigm designed to optimize reasoning efficiently while preserving learned prior knowledge. SPoT consists of: (1) a data rectification pipeline that employs an Oracle to surgically correct erroneous steps via minimal edits, generating data proximal to the model's distribution; and (2) a reward-based binary cross-entropy objective. Unlike the relative ranking in DPO, this objective treats reasoning correctness as a binary classification problem, enforcing decoupled supervision signals. Empirically, with only 4k rectified math data pairs, SPoT improves Qwen3-8B's accuracy by 6.2% on average across in-domain and OOD tasks, requiring mere...