Abductive reasoning aims to make the most likely inference for a given set of incomplete observations. In this work, we propose a new task called abductive action inference, in which given a situation, the model answers the question `what actions were executed by the human in order to arrive in the current state?'. Given a state, we investigate three abductive inference problems: action set prediction, action sequence prediction, and abductive action verification. We benchmark several SOTA models such as Transformers, Graph neural networks, CLIP, BLIP, end-to-end trained Slow-Fast, and Resnet50-3D models. Our newly proposed object-relational BiGED model outperforms all other methods on this challenging task on the Action Genome dataset. Codes will be made available.
翻译：摘要：Abductive推理旨在对给定的不完整观察进行最可能的推断。在本文中，我们提出了一个新的任务，称为Abductive行动推理，其中模型在给定情况下回答问题“人类执行了哪些行动，以达到当前状态？”在给定状态下，我们研究了三个Abductive 推理问题：行动集预测、行动序列预测和Abductive行动验证。我们对几个最先进的模型进行了基准测试，例如Transformers、Graph神经网络、CLIP、BLIP、端到端训练的Slow-Fast和Resnet50-3D模型。我们新提出的对象关系BiGED模型在Action Genome数据集上表现出色，胜过所有其他方法。代码将会公开发布。
相关内容
微信扫码咨询专知VIP会员