Pushing is an essential non-prehensile manipulation skill used for tasks ranging from pre-grasp manipulation to scene rearrangement, reasoning about object relations in the scene, and thus pushing actions have been widely studied in robotics. The effective use of pushing actions often requires an understanding of the dynamics of the manipulated objects and adaptation to the discrepancies between prediction and reality. For this reason, effect prediction and parameter estimation with pushing actions have been heavily investigated in the literature. However, current approaches are limited because they either model systems with a fixed number of objects or use image-based representations whose outputs are not very interpretable and quickly accumulate errors. In this paper, we propose a graph neural network based framework for effect prediction and parameter estimation of pushing actions by modeling object relations based on contacts or articulations. Our framework is validated both in real and simulated environments containing different shaped multi-part objects connected via different types of joints and objects with different masses, and it outperforms image-based representations on physics prediction. Our approach enables the robot to predict and adapt the effect of a pushing action as it observes the scene. It can also be used for tool manipulation with never-seen tools. Further, we demonstrate 6D effect prediction in the lever-up action in the context of robot-based hard-disk disassembly.
翻译:推动是一种重要的非痛苦的操纵技能,用于从预抓操纵到现场重新排列等任务,对现场物体关系进行推理,因此对推进行动进行了广泛的研究。 有效使用推动行动往往需要了解被操纵物体的动态,并适应预测与现实之间的差异。 为此,文献对影响预测和推动行动参数估计进行了大量调查。然而,目前的方法是有限的,因为它们要么是具有固定数量物体的模型系统,要么是使用其产出不易解释和迅速积累错误的图像显示器。在本文件中,我们提出了一个基于图形的神经网络框架,用于根据接触或表达方式模拟物体关系模型对推动行动的效果预测和参数估计。我们的框架在真实和模拟环境中都得到验证,其中包含不同形状的多部分物体,通过不同种类的连接和不同质量的组合和物体进行联系,而且它比物理预测的图像表示法更不完善。我们的方法使机器人能够预测和调整推动行动的效果,因为其输出结果不易理解。我们还可以用六维的机器人工具进行工具操纵。我们无法预见的机器人。