Dexterous manipulation with a multi-finger hand is one of the most challenging problems in robotics. While recent progress in imitation learning has largely improved the sample efficiency compared to Reinforcement Learning, the learned policy can hardly generalize to manipulate novel objects, given limited expert demonstrations. In this paper, we propose to learn dexterous manipulation using large-scale demonstrations with diverse 3D objects in a category, which are generated from a human grasp affordance model. This generalizes the policy to novel object instances within the same category. To train the policy, we propose a novel imitation learning objective jointly with a geometric representation learning objective using our demonstrations. By experimenting with relocating diverse objects in simulation, we show that our approach outperforms baselines with a large margin when manipulating novel objects. We also ablate the importance on 3D object representation learning for manipulation. We include videos, code, and additional information on the project website - https://kristery.github.io/ILAD/ .
翻译:多指手巧操作是机器人中最具挑战性的问题之一。 虽然在模仿学习方面最近的进展与加强学习相比,大大提高了样本效率,但由于专家演示有限,学习的政策很难概括地操作新物品。 在本文中,我们提议学习利用人类掌握的发配模式产生的类别中各种三维物体的大规模操纵。这把政策概括到在同一类别中新增加对象实例。为了培训政策,我们提议了一种新颖的模仿学习目标,同时利用我们的演示来进行几何代表学习。通过在模拟中实验迁移各种物体,我们表明我们的方法在操纵新物品时大大超越了基线。我们还放弃了3D对象代表学习的重要性。我们把视频、代码和更多项目网站 - https://kristery.github.io/ILAD/ 上的信息都包括视频、代码和补充信息。