[2510.15495] OffSim: Offline Simulator for Model-based Offline Inverse Reinforcement Learning

Computer Science > Machine Learning arXiv:2510.15495 (cs) This paper has been withdrawn by Woo-Jin Ahn [Submitted on 17 Oct 2025 (v1), last revised 25 Mar 2026 (this version, v2)] Title:OffSim: Offline Simulator for Model-based Offline Inverse Reinforcement Learning Authors:Woo-Jin Ahn, Sang-Ryul Baek, Yong-Jun Lee, Hyun-Duck Choi, Myo-Taeg Lim View a PDF of the paper titled OffSim: Offline Simulator for Model-based Offline Inverse Reinforcement Learning, by Woo-Jin Ahn and 4 other authors No PDF available, click to view other formats Abstract:Reinforcement learning algorithms typically utilize an interactive simulator (i.e., environment) with a predefined reward function for policy training. Developing such simulators and manually defining reward functions, however, is often time-consuming and labor-intensive. To address this, we propose an Offline Simulator (OffSim), a novel model-based offline inverse reinforcement learning (IRL) framework, to emulate environmental dynamics and reward structure directly from expert-generated state-action trajectories. OffSim jointly optimizes a high-entropy transition model and an IRL-based reward function to enhance exploration and improve the generalizability of the learned reward. Leveraging these learned components, OffSim can subsequently train a policy offline without further interaction with the real environment. Additionally, we introduce OffSim$^+$, an extension that incorporates a marginal reward for multi-dataset settings to ...