![[2602.13197] Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videos](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.13197] Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videos
Summary
This article presents a framework called Perceive-Simulate-Imitate (PSI) for training robots to learn manipulation skills from human videos, focusing on grasping and post-grasp motions.
Why It Matters
The research addresses a significant challenge in robotics: enabling robots to learn complex manipulation tasks from human demonstrations. By utilizing human video data and a modular policy design, the PSI framework enhances the efficiency and robustness of robot learning, which could lead to advancements in automation and human-robot interaction.
Key Takeaways
- The PSI framework allows robots to learn manipulation skills from human videos without requiring robot-specific data.
- A modular policy design helps in generating task-compatible grasps, improving the robot's performance.
- The approach shows significant improvements in learning efficiency and robustness compared to traditional methods.
Computer Science > Robotics arXiv:2602.13197 (cs) [Submitted on 13 Feb 2026] Title:Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videos Authors:Albert J. Zhai, Kuo-Hao Zeng, Jiasen Lu, Ali Farhadi, Shenlong Wang, Wei-Chiu Ma View a PDF of the paper titled Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videos, by Albert J. Zhai and 5 other authors View PDF HTML (experimental) Abstract:The ability to learn manipulation skills by watching videos of humans has the potential to unlock a new source of highly scalable data for robot learning. Here, we tackle prehensile manipulation, in which tasks involve grasping an object before performing various post-grasp motions. Human videos offer strong signals for learning the post-grasp motions, but they are less useful for learning the prerequisite grasping behaviors, especially for robots without human-like hands. A promising way forward is to use a modular policy design, leveraging a dedicated grasp generator to produce stable grasps. However, arbitrary stable grasps are often not task-compatible, hindering the robot's ability to perform the desired downstream motion. To address this challenge, we present Perceive-Simulate-Imitate (PSI), a framework for training a modular manipulation policy using human video motion data processed by paired grasp-trajectory filtering in simulation. This simulation step extends the trajectory data with grasp suitability labels, which allows ...
