[2603.03280] How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference

Computer Science > Robotics arXiv:2603.03280 (cs) [Submitted on 3 Mar 2026] Title:How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference Authors:Toru Lin, Shuying Deng, Zhao-Heng Yin, Pieter Abbeel, Jitendra Malik View a PDF of the paper titled How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference, by Toru Lin and 4 other authors View PDF HTML (experimental) Abstract:Many essential manipulation tasks - such as food preparation, surgery, and craftsmanship - remain intractable for autonomous robots. These tasks are characterized not only by contact-rich, force-sensitive dynamics, but also by their "implicit" success criteria: unlike pick-and-place, task quality in these domains is continuous and subjective (e.g. how well a potato is peeled), making quantitative evaluation and reward engineering difficult. We present a learning framework for such tasks, using peeling with a knife as a representative example. Our approach follows a two-stage pipeline: first, we learn a robust initial policy via force-aware data collection and imitation learning, enabling generalization across object variations; second, we refine the policy through preference-based finetuning using a learned reward model that combines quantitative task metrics with qualitative human feedback, aligning policy behavior with human notions of task quality. Using only 50-200 peeling trajectories, our system achieves over 90% average success rates on challengin...