![[2602.15828] Dex4D: Task-Agnostic Point Track Policy for Sim-to-Real Dexterous Manipulation](https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png){kind=logo}
[2602.15828] Dex4D: Task-Agnostic Point Track Policy for Sim-to-Real Dexterous Manipulation
Summary
The Dex4D framework enables task-agnostic dexterous manipulation by using simulation to learn generalist policies that can be applied to diverse real-world tasks without fine-tuning.
Why It Matters
Dexterous manipulation is crucial for robotics, yet developing adaptable skills remains challenging. Dex4D addresses this by facilitating zero-shot transfer of learned skills from simulation to real-world applications, enhancing efficiency and scalability in robotic tasks.
Key Takeaways
- Dex4D learns a domain-agnostic 3D point track policy for manipulation.
- It allows zero-shot deployment of skills to real-world tasks without fine-tuning.
- The framework shows strong generalization to novel objects and environments.
- Extensive experiments demonstrate consistent improvements over prior methods.
- Dex4D enhances the scalability of robotic manipulation tasks.
Computer Science > Robotics arXiv:2602.15828 (cs) [Submitted on 17 Feb 2026] Title:Dex4D: Task-Agnostic Point Track Policy for Sim-to-Real Dexterous Manipulation Authors:Yuxuan Kuang, Sungjae Park, Katerina Fragkiadaki, Shubham Tulsiani View a PDF of the paper titled Dex4D: Task-Agnostic Point Track Policy for Sim-to-Real Dexterous Manipulation, by Yuxuan Kuang and 3 other authors View PDF HTML (experimental) Abstract:Learning generalist policies capable of accomplishing a plethora of everyday tasks remains an open challenge in dexterous manipulation. In particular, collecting large-scale manipulation data via real-world teleoperation is expensive and difficult to scale. While learning in simulation provides a feasible alternative, designing multiple task-specific environments and rewards for training is similarly challenging. We propose Dex4D, a framework that instead leverages simulation for learning task-agnostic dexterous skills that can be flexibly recomposed to perform diverse real-world manipulation tasks. Specifically, Dex4D learns a domain-agnostic 3D point track conditioned policy capable of manipulating any object to any desired pose. We train this 'Anypose-to-Anypose' policy in simulation across thousands of objects with diverse pose configurations, covering a broad space of robot-object interactions that can be composed at test time. At deployment, this policy can be zero-shot transferred to real-world tasks without finetuning, simply by prompting it with desi...
