[2603.02045] Expanding LLM Agent Boundaries with Strategy-Guided Exploration

Computer Science > Machine Learning arXiv:2603.02045 (cs) [Submitted on 2 Mar 2026] Title:Expanding LLM Agent Boundaries with Strategy-Guided Exploration Authors:Andrew Szot, Michael Kirchhof, Omar Attia, Alexander Toshev View a PDF of the paper titled Expanding LLM Agent Boundaries with Strategy-Guided Exploration, by Andrew Szot and 3 other authors View PDF HTML (experimental) Abstract:Reinforcement learning (RL) has demonstrated notable success in post-training large language models (LLMs) as agents for tasks such as computer use, tool calling, and coding. However, exploration remains a central challenge in RL for LLM agents, especially as they operate in language-action spaces with complex observations and sparse outcome rewards. In this work, we address exploration for LLM agents by leveraging the ability of LLMs to plan and reason in language about the environment to shift exploration from low-level actions to higher-level language strategies. We thus propose Strategy-Guided Exploration (SGE), which first generates a concise natural-language strategy that describes what to do to make progress toward the goal, and then generates environment actions conditioned on that strategy. By exploring in the space of strategies rather than the space of actions, SGE induces structured and diverse exploration that targets different environment outcomes. To increase strategy diversity during RL, SGE introduces mixed-temperature sampling, which explores diverse strategies in paralle...