Abductive reasoning aims to make the most likely inference for a given set of incomplete observations. In this work, we propose a new task called abductive action inference, in which given a situation, the model answers the question `what actions were executed by the human in order to arrive in the current state?'. Given a state, we investigate three abductive inference problems: action set prediction, action sequence prediction, and abductive action verification. We benchmark several SOTA models such as Transformers, Graph neural networks, CLIP, BLIP, end-to-end trained Slow-Fast, and Resnet50-3D models. Our newly proposed object-relational BiGED model outperforms all other methods on this challenging task on the Action Genome dataset. Codes will be made available.
翻译：摘要：Abductive推理旨在对给定的不完整观察做出最有可能的推断。在这项工作中，我们提出了一个新的任务，称为Abductive行动推理，在给定情况的情况下，模型回答问题“人类执行了哪些行动以到达当前状态？”。给定状态，我们研究了三个Abductive推理问题：行动集预测，行动序列预测和Abductive行动验证。我们对多个最新模型进行了基准测试，如：Transformers, Graph神经网络, CLIP, BLIP, end-to-end trained Slow-Fast, 以及 Resnet50-3D模型。我们新提出的BiGED对象关系模型在Action Genome数据集上的表现优于所有其他方法。代码将提供。
相关内容
微信扫码咨询专知VIP会员