[2604.08958] WOMBET: World Model-based Experience Transfer for Robust and Sample-efficient Reinforcement Learning

Computer Science > Machine Learning arXiv:2604.08958 (cs) [Submitted on 10 Apr 2026] Title:WOMBET: World Model-based Experience Transfer for Robust and Sample-efficient Reinforcement Learning Authors:Mintae Kim, Koushil Sreenath View a PDF of the paper titled WOMBET: World Model-based Experience Transfer for Robust and Sample-efficient Reinforcement Learning, by Mintae Kim and 1 other authors View PDF HTML (experimental) Abstract:Reinforcement learning (RL) in robotics is often limited by the cost and risk of data collection, motivating experience transfer from a source task to a target task. Offline-to-online RL leverages prior data but typically assumes a given fixed dataset and does not address how to generate reliable data for transfer. We propose \textit{World Model-based Experience Transfer} (WOMBET), a framework that jointly generates and utilizes prior data. WOMBET learns a world model in the source task and generates offline data via uncertainty-penalized planning, followed by filtering trajectories with high return and low epistemic uncertainty. It then performs online fine-tuning in the target task using adaptive sampling between offline and online data, enabling a stable transition from prior-driven initialization to task-specific adaptation. We show that the uncertainty-penalized objective provides a lower bound on the true return and derive a finite-sample error decomposition capturing distribution mismatch and approximation error. Empirically, WOMBET improve...