[2603.04948] $\nabla$-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space

Computer Science > Machine Learning arXiv:2603.04948 (cs) [Submitted on 5 Mar 2026] Title:$\nabla$-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space Authors:Peihao Wang, Ruisi Cai, Zhen Wang, Hongyuan Mei, Qiang Liu, Pan Li, Zhangyang Wang View a PDF of the paper titled $\nabla$-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space, by Peihao Wang and 6 other authors View PDF HTML (experimental) Abstract:Scaling inference-time compute for Large Language Models (LLMs) has unlocked unprecedented reasoning capabilities. However, existing inference-time scaling methods typically rely on inefficient and suboptimal discrete search algorithms or trial-and-error prompting to improve the online policy. In this paper, we propose $\nabla$-Reasoner, an iterative generation framework that integrates differentiable optimization over token logits into the decoding loop to refine the policy on the fly. Our core component, Differentiable Textual Optimization (DTO), leverages gradient signals from both the LLM's likelihood and a reward model to refine textual representations. $\nabla$-Reasoner further incorporates rejection sampling and acceleration design to robustify and speed up decoding. Theoretically, we show that performing inference-time gradient descent in the sample space to maximize reward is dual to aligning an LLM policy via KL-regularized reinforcement learning. Empirically, $\nabla$-Reasoner achieves over 20% accuracy improvement on a chal...