Autonomous robots can benefit greatly from human-provided semantic characterizations of uncertain task environments and states. However, the development of integrated strategies which let robots model, communicate, and act on such 'soft data' remains challenging. Here, the Human Assisted Robotic Planning and Sensing (HARPS) framework is presented for active semantic sensing and planning in human-robot teams to address these gaps by formally combining the benefits of online sampling-based POMDP policies, multimodal semantic interaction, and Bayesian data fusion. This approach lets humans opportunistically impose model structure and extend the range of semantic soft data in uncertain environments by sketching and labeling arbitrary landmarks across the environment. Dynamic updating of the environment model while during search allows robotic agents to actively query humans for novel and relevant semantic data, thereby improving beliefs of unknown environments and states for improved online planning. Simulations of a UAV-enabled target search application in a large-scale partially structured environment show significant improvements in time and belief state estimates required for interception versus conventional planning based solely on robotic sensing. Human subject studies in the same environment (n = 36) demonstrate an average doubling in dynamic target capture rate compared to the lone robot case, and highlight the robustness of active probabilistic reasoning and semantic sensing over a range of user characteristics and interaction modalities.
翻译:自主机器人可以从人类提供的不确定任务环境和状态的语义描述中受益匪浅。然而,将机器人模型、通信和行动与此类“软数据”紧密结合起来的综合策略仍然具有挑战性。本文介绍了用于人-机器人团队中主动语义感知和规划的Human Assisted Robotic Planning and Sensing(HARPS)框架,以通过正式结合在线基于采样的POMDP策略、多模态语义交互和贝叶斯数据融合来解决这些差距。这种方法让人类有机会在不确定的环境中通过对环境中的任意地标进行素描和标记来施加模型结构和扩展语义软数据的范围。在搜索期间动态更新环境模型,使机器人代理能够主动询问人类获得新颖和相关的语义数据,从而提高未知环境和状态的信念以改进在线规划。在大规模部分结构化环境下进行的UAV启用的目标搜索应用程序的模拟显示,与仅基于机器人感知的传统规划相比,所需的截击时间和信念状态估计有显着改进。与单一机器人情况相比,同一环境中的人类受试者研究(n = 36)表明动态目标捕获率平均翻倍,并突显了积极概率推理和语义感知在一系列用户特征和交互模式上的稳健性。