[2603.21558] Stabilizing Iterative Self-Training with Verified Reasoning via Symbolic Recursive Self-Alignment

Computer Science > Artificial Intelligence arXiv:2603.21558 (cs) [Submitted on 23 Mar 2026] Title:Stabilizing Iterative Self-Training with Verified Reasoning via Symbolic Recursive Self-Alignment Authors:Xinyu Zhang View a PDF of the paper titled Stabilizing Iterative Self-Training with Verified Reasoning via Symbolic Recursive Self-Alignment, by Xinyu Zhang View PDF HTML (experimental) Abstract:Recursive self-improvement--where a model iteratively trains on its own outputs--promises sustained capability growth but faces a fundamental obstacle: recursive drift. As models train on self-generated data across multiple iterations, errors in intermediate reasoning compound, leading to mode collapse and performance degradation. We propose Neuro-Symbolic Recursive Self-Alignment (NSRSA), which stabilizes iterative self-training by embedding a symbolic verification subsystem that gates training data quality at the reasoning step level. Unlike outcome-only filtering (which admits "lucky guesses" with flawed reasoning), NSRSA verifies each arithmetic operation via sympy, checks logical flow consistency across reasoning steps, and enforces domain constraints. We evaluate NSRSA on GSM8K using Qwen3-4B-Thinking across 5 self-training iterations under five conditions: no verification, outcome verification, majority voting, full NSRSA symbolic verification, and NSRSA with DPO. Our filtering analysis shows that NSRSA rejects approximately 34% of correct-answer solutions that pass outcome ...