[2507.18014] Predictive Scaling Laws for Efficient GRPO Training of Large Reasoning Models

Computer Science > Machine Learning arXiv:2507.18014 (cs) [Submitted on 24 Jul 2025 (v1), last revised 19 Mar 2026 (this version, v3)] Title:Predictive Scaling Laws for Efficient GRPO Training of Large Reasoning Models Authors:Datta Nimmaturi, Vaishnavi Bhargava, Rajat Ghosh, Johnu George, Debojyoti Dutta View a PDF of the paper titled Predictive Scaling Laws for Efficient GRPO Training of Large Reasoning Models, by Datta Nimmaturi and 4 other authors View PDF HTML (experimental) Abstract:Fine-tuning large language models (LLMs) for reasoning tasks using reinforcement learning methods like Group Relative Policy Optimization (GRPO) is computationally expensive. To address this, we propose a predictive framework that models training dynamics and helps optimize resource usage. Through experiments on Llama and Qwen models (3B 8B), we derive an empirical scaling law based on model size, initial performance, and training progress. This law predicts reward trajectories and identifies three consistent training phases: slow start, rapid improvement, and plateau. We find that training beyond certain number of an epoch offers little gain, suggesting earlier stopping can significantly reduce compute without sacrificing performance. Our approach generalizes across model types, providing a practical guide for efficient GRPO-based fine-tuning. Subjects: Machine Learning (cs.LG) Cite as: arXiv:2507.18014 [cs.LG]   (or arXiv:2507.18014v3 [cs.LG] for this version)   https://doi.org/10.48550...