[2601.08427] Silence the Judge: Reinforcement Learning with Self-Verifier via Latent Geometric Clustering

Computer Science > Computation and Language arXiv:2601.08427 (cs) [Submitted on 13 Jan 2026 (v1), last revised 1 Mar 2026 (this version, v2)] Title:Silence the Judge: Reinforcement Learning with Self-Verifier via Latent Geometric Clustering Authors:Nonghai Zhang, Weitao Ma, Zhanyu Ma, Jun Xu, Jiuchong Gao, Jinghua Hao, Renqing He, Jingwen Xu View a PDF of the paper titled Silence the Judge: Reinforcement Learning with Self-Verifier via Latent Geometric Clustering, by Nonghai Zhang and 7 other authors View PDF HTML (experimental) Abstract:Group Relative Policy Optimization (GRPO) significantly enhances the reasoning performance of Large Language Models (LLMs). However, this success heavily relies on expensive external verifiers or human rules. Such dependency not only leads to significant computational costs and training latency, but also yields sparse rewards that hinder optimization efficiency. To address these challenges, we propose Latent-GRPO, a framework that derives intrinsic rewards directly from latent space geometry. Crucially, our empirical analysis reveals a compelling geometric property: terminal token representations of correct reasoning trajectories form dense clusters with high intra-class similarity, whereas incorrect trajectories remain scattered as outliers. In light of this discovery, we introduce the Iterative Robust Centroid Estimation (IRCE) algorithm, which generates dense, continuous rewards by mitigating magnitude fluctuations via spherical project...