![[2602.13865] Enabling Option Learning in Sparse Rewards with Hindsight Experience Replay](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.13865] Enabling Option Learning in Sparse Rewards with Hindsight Experience Replay
Summary
This paper introduces MOC-HER and 2HER, methods that enhance Hierarchical Reinforcement Learning by improving performance in sparse reward environments, particularly in robotic manipulation tasks.
Why It Matters
The research addresses significant challenges in Hierarchical Reinforcement Learning (HRL) by proposing novel methods that improve learning efficiency in environments with sparse rewards. This has implications for advancing AI applications in robotics and complex task automation, where traditional methods struggle.
Key Takeaways
- MOC-HER integrates Hindsight Experience Replay to improve learning in sparse reward scenarios.
- 2HER introduces dual objectives for goal relabeling, enhancing task success rates.
- Experimental results show MOC-2HER achieves up to 90% success in robotic tasks, significantly outperforming prior methods.
Computer Science > Artificial Intelligence arXiv:2602.13865 (cs) [Submitted on 14 Feb 2026] Title:Enabling Option Learning in Sparse Rewards with Hindsight Experience Replay Authors:Gabriel Romio, Mateus Begnini Melchiades, Bruno Castro da Silva, Gabriel de Oliveira Ramos View a PDF of the paper titled Enabling Option Learning in Sparse Rewards with Hindsight Experience Replay, by Gabriel Romio and 3 other authors View PDF HTML (experimental) Abstract:Hierarchical Reinforcement Learning (HRL) frameworks like Option-Critic (OC) and Multi-updates Option Critic (MOC) have introduced significant advancements in learning reusable options. However, these methods underperform in multi-goal environments with sparse rewards, where actions must be linked to temporally distant outcomes. To address this limitation, we first propose MOC-HER, which integrates the Hindsight Experience Replay (HER) mechanism into the MOC framework. By relabeling goals from achieved outcomes, MOC-HER can solve sparse reward environments that are intractable for the original MOC. However, this approach is insufficient for object manipulation tasks, where the reward depends on the object reaching the goal rather than on the agent's direct interaction. This makes it extremely difficult for HRL agents to discover how to interact with these objects. To overcome this issue, we introduce Dual Objectives Hindsight Experience Replay (2HER), a novel extension that creates two sets of virtual goals. In addition to re...
