[2506.06658] Self-Improving Loops for Visual Robotic Planning

Computer Science > Robotics arXiv:2506.06658 (cs) [Submitted on 7 Jun 2025 (v1), last revised 3 Mar 2026 (this version, v2)] Title:Self-Improving Loops for Visual Robotic Planning Authors:Calvin Luo, Zilai Zeng, Mingxi Jia, Yilun Du, Chen Sun View a PDF of the paper titled Self-Improving Loops for Visual Robotic Planning, by Calvin Luo and 4 other authors View PDF HTML (experimental) Abstract:Video generative models trained on expert demonstrations have been utilized as performant text-conditioned visual planners for solving robotic tasks. However, generalization to unseen tasks remains a challenge. Whereas improved generalization may be facilitated by leveraging learned prior knowledge from additional pre-collected offline data sources, such as web-scale video datasets, in the era of experience we aim to design agents that can continuously improve in an online manner from self-collected behaviors. In this work we thus propose the Self-Improving Loops for Visual Robotic Planning (SILVR), where an in-domain video model iteratively updates itself on self-produced trajectories, and steadily improves its performance for a specified task of interest. We apply SILVR to a diverse suite of MetaWorld tasks, as well as two manipulation tasks on a real robot arm, and find that performance improvements continuously emerge over multiple iterations for novel tasks unseen during initial in-domain video model training. We demonstrate that SILVR is robust in the absence of human-provided g...