Object affordance is an important concept in hand-object interaction, providing information on action possibilities based on human motor capacity and objects' physical property thus benefiting tasks such as action anticipation and robot imitation learning. However, the definition of affordance in existing datasets often: 1) mix up affordance with object functionality; 2) confuse affordance with goal-related action; and 3) ignore human motor capacity. This paper proposes an efficient annotation scheme to address these issues by combining goal-irrelevant motor actions and grasp types as affordance labels and introducing the concept of mechanical action to represent the action possibilities between two objects. We provide new annotations by applying this scheme to the EPIC-KITCHENS dataset and test our annotation with tasks such as affordance recognition, hand-object interaction hotspots prediction, and cross-domain evaluation of affordance. The results show that models trained with our annotation can distinguish affordance from other concepts, predict fine-grained interaction possibilities on objects, and generalize through different domains.
翻译:物体承受能力是人工物体相互作用中的一个重要概念,根据人的运动能力和物体的物理属性提供关于行动可能性的信息,从而有利于行动预期和机器人模拟学习等任务。然而,现有数据集中给付能力的定义往往:(1) 与物体功能混在一起;(2) 将给付与目标相关行动混淆;和(3) 忽视人的运动能力。本文件提出一个有效的说明计划,通过将目标相关运动动作和捕捉类型合并为给付标签来解决这些问题,并引入机械行动概念来代表两个物体之间的行动可能性。我们提供了新的说明,将这一计划应用到 EPIC-KITCHENS 数据集中,测试我们用给付识别、手对对象交互热点预测和对给付能力跨部评估等任务所作的说明。结果显示,经过我们培训的模型可以区分给付能力与其他概念,预测对象的精细的相互作用可能性,并通过不同领域加以普及。