![[2602.20220] What Matters for Simulation to Online Reinforcement Learning on Real Robots](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.20220] What Matters for Simulation to Online Reinforcement Learning on Real Robots
Summary
This paper explores design choices that enhance online reinforcement learning (RL) on physical robots, presenting findings from 100 training runs across various platforms.
Why It Matters
Understanding the critical design choices for online RL on real robots is essential for practitioners aiming to implement effective and efficient learning systems. This research provides empirical evidence that can reduce engineering effort and improve outcomes in robotic applications.
Key Takeaways
- Identifies harmful defaults in common RL practices.
- Offers robust design choices that enhance stability in learning.
- Presents the first large-sample empirical study in this area.
- Aims to lower engineering effort for deploying online RL.
- Highlights the importance of systematic ablation in RL research.
Computer Science > Robotics arXiv:2602.20220 (cs) [Submitted on 23 Feb 2026] Title:What Matters for Simulation to Online Reinforcement Learning on Real Robots Authors:Yarden As, Dhruva Tirumala, René Zurbrügg, Chenhao Li, Stelian Coros, Andreas Krause, Markus Wulfmeier View a PDF of the paper titled What Matters for Simulation to Online Reinforcement Learning on Real Robots, by Yarden As and 6 other authors View PDF HTML (experimental) Abstract:We investigate what specific design choices enable successful online reinforcement learning (RL) on physical robots. Across 100 real-world training runs on three distinct robotic platforms, we systematically ablate algorithmic, systems, and experimental decisions that are typically left implicit in prior work. We find that some widely used defaults can be harmful, while a set of robust, readily adopted design choices within standard RL practice yield stable learning across tasks and hardware. These results provide the first large-sample empirical study of such design choices, enabling practitioners to deploy online RL with lower engineering effort. Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI) Cite as: arXiv:2602.20220 [cs.RO] (or arXiv:2602.20220v1 [cs.RO] for this version) https://doi.org/10.48550/arXiv.2602.20220 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Yarden As [view email] [v1] Mon, 23 Feb 2026 10:34:15 UTC (17,102 KB) Full-text links: Access Paper: Vie...