In this work, we tackle the problem of learning universal robotic dexterous grasping from a point cloud observation under a table-top setting. The goal is to grasp and lift up objects in high-quality and diverse ways and generalize across hundreds of categories and even the unseen. Inspired by successful pipelines used in parallel gripper grasping, we split the task into two stages: 1) grasp proposal (pose) generation and 2) goal-conditioned grasp execution. For the first stage, we propose a novel probabilistic model of grasp pose conditioned on the point cloud observation that factorizes rotation from translation and articulation. Trained on our synthesized large-scale dexterous grasp dataset, this model enables us to sample diverse and high-quality dexterous grasp poses for the object in the point cloud. For the second stage, we propose to replace the motion planning used in parallel gripper grasping with a goal-conditioned grasp policy, due to the complexity involved in dexterous grasping execution. Note that it is very challenging to learn this highly generalizable grasp policy that only takes realistic inputs without oracle states. We thus propose several important innovations, including state canonicalization, object curriculum, and teacher-student distillation. Integrating the two stages, our final pipeline becomes the first to achieve universal generalization for dexterous grasping, demonstrating an average success rate of more than 60% on thousands of object instances, which significantly out performs all baselines, meanwhile showing only a minimal generalization gap.
翻译:在这项工作中,我们从一张桌面设置的云层观察点上学习通用机器人宽度捕捉的问题。目标是以高质量和多样的方式抓住和提升物体,并在数百个类别甚至看不见的类别中推广。在同时抓住成功管道的启发下,我们将任务分成两个阶段:1)抓住建议(应用)产生,2)有目标限制的抓住执行。在第一阶段,我们提出了一个新的概率掌握模式,它以云层观测点为条件,将翻译和表达的轮换作为因素。我们用我们综合的大型宽度捕获数据集培训了这些物体,从而使我们能够抽样展示多样化和高质量的宽度捕捉,成为了点云中对象。在第二阶段,我们提议用一个有目标限制的抓住政策来取代同时捕捉的运动规划。我们提出了一种具有目标限制的抓住政策。我们提出一个具有新颖复杂性的抓住模式,只是以极低度的云层观测点为条件,将云层从翻译和表达出从实际输入或达到目的的目标。这个模式使我们能够对广度的广度捕获目标进行训练。因此,我们提出了多种重要、高度的宽度的深度的深度的深度理解。我们逐渐推进的基线化的阶段,包括了整个师级阶段,从而逐渐地展示了我们整个阶段的进度化为最后阶段。我们进入两个阶段的阶段。我们逐渐地展示了整个阶段的阶段,包括了整个阶段的学习了整个阶段。我们整个阶段的进度级级级级级级级级级的阶段,我们学习了整个阶段。我级级级级级级级级级级级级级级级级级级进进进进进进进进进进进进进进进进进进进进进进进进进进进到最后的进进进进进进进进进进进进进进进到最后的进进进到最后的阶段。进进进到最后的进进到最后的阶段。进到最后的阶段。进到最后的阶段。进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进进</s>