[2604.03675] PRAISE: Prefix-Based Rollout Reuse in Agentic Search Training

Computer Science > Artificial Intelligence arXiv:2604.03675 (cs) [Submitted on 4 Apr 2026] Title:PRAISE: Prefix-Based Rollout Reuse in Agentic Search Training Authors:Erhan Zhang, Yiqun Chen, Zechun Niu, Wei Yang, Xiaochi Wei, Yan Gao, Yi Wu, Yao Hu, Jiaxin Mao View a PDF of the paper titled PRAISE: Prefix-Based Rollout Reuse in Agentic Search Training, by Erhan Zhang and 8 other authors View PDF HTML (experimental) Abstract:In agentic search, large language models (LLMs) are trained to perform multi-turn retrieval and reasoning for complex tasks such as multi-hop question answering (QA). However, current search-based Reinforcement Learning (RL) methods suffer from two core limitations: expensive long-horizon rollouts are under-utilized during training, and supervision is typically available only at the final answer, resulting in severe reward sparsity. We present Prefix-based Rollout reuse for Agentic search with Intermediate Step rEwards (PRAISE), a framework for improving both data efficiency and credit assignment in agentic search training. Given a complete search trajectory, PRAISE extracts prefix states at different search turns, elicits intermediate answers from them, and uses these prefixes both to construct additional training trajectories and to derive step-level rewards from performance differences across prefixes. Our method uses a single shared model for both search policy learning and prefix answer evaluation, enabling joint optimization without extra human a...