For a robot to repair its own error, it must first know it has made a mistake. One way that people detect errors is from the implicit reactions from bystanders -- their confusion, smirks, or giggles clue us in that something unexpected occurred. To enable robots to detect and act on bystander responses to task failures, we developed a novel method to elicit bystander responses to human and robot errors. Using 46 different stimulus videos featuring a variety of human and machine task failures, we collected a total of 2452 webcam videos of human reactions from 54 participants. To test the viability of the collected data, we used the bystander reaction dataset as input to a deep-learning model, BADNet, to predict failure occurrence. We tested different data labeling methods and learned how they affect model performance, achieving precisions above 90%. We discuss strategies to model bystander reactions and predict failure and how this approach can be used in real-world robotic deployments to detect errors and improve robot performance. As part of this work, we also contribute with the "Bystander Affect Detection" (BAD) dataset of bystander reactions, supporting the development of better prediction models.
翻译:机器人要修复自己的错误,首先必须知道自己犯了错误。 人们发现错误的一种方法是从旁观者暗中反应中发现错误 -- -- 他们的困惑、笑笑或笑笑告诉我们出乎意料的事情发生。为了使机器人能够探测和根据旁观者对任务失败的反应采取行动,我们开发了一种新颖的方法来诱使旁观者对人类和机器人错误作出反应。我们用46个具有各种人类和机器任务失败特征的不同刺激视频,共收集了54名参与者对人类反应的2452个网络摄像头视频。为了测试所收集的数据的可行性,我们用旁观者反应数据集作为深学习模型(BADNet)的输入,以预测失败发生。我们测试了不同的数据标签方法,并学习了它们如何影响模型性能,达到90%以上的精确度。我们讨论了模拟旁观者反应和预测失败的战略,以及如何在现实世界机器人部署中采用这种方法来检测错误并改进机器人性能。作为这项工作的一部分,我们还利用“旁观者Affect检测” (BADD) 数据集,以支持更精确的预测模型的发展。</s>