[2510.13358] Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot Control

Computer Science > Robotics arXiv:2510.13358 (cs) [Submitted on 15 Oct 2025 (v1), last revised 27 Feb 2026 (this version, v2)] Title:Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot Control Authors:Shingo Ayabe, Hiroshi Kera, Kazuhiko Kawamoto View a PDF of the paper titled Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot Control, by Shingo Ayabe and 2 other authors View PDF HTML (experimental) Abstract:Offline reinforcement learning enables sample-efficient policy acquisition without risky online interaction, yet policies trained on static datasets remain brittle under action-space perturbations such as actuator faults. This study introduces an offline-to-online framework that trains policies on clean data and then performs adversarial fine-tuning, where perturbations are injected into executed actions to induce compensatory behavior and improve resilience. A performance-aware curriculum further adjusts the perturbation probability during training via an exponential-moving-average signal, balancing robustness and stability throughout the learning process. Experiments on continuous-control locomotion tasks demonstrate that the proposed method consistently improves robustness over offline-only baselines and converges faster than training from scratch. Matching the fine-tuning and evaluation conditions yields the strongest robustness to action-space perturbations, while the adaptive curriculum strategy m...