An important prerequisite for the reliability and robustness of a service robot is ensuring the robot's correct behavior when it performs various tasks of interest. Extensive testing is one established approach for ensuring behavioural correctness; this becomes even more important with the integration of learning-based methods into robot software architectures, as there are often no theoretical guarantees about the performance of such methods in varying scenarios. In this paper, we aim towards evaluating the correctness of robot behaviors in tabletop manipulation through automatic generation of simulated test scenarios in which a robot assesses its performance using property-based testing. In particular, key properties of interest for various robot actions are encoded in an action ontology and are then verified and validated within a simulated environment. We evaluate our framework with a Toyota Human Support Robot (HSR) which is tested in a Gazebo simulation. We show that our framework can correctly and consistently identify various failed actions in a variety of randomised tabletop manipulation scenarios, in addition to providing deeper insights into the type and location of failures for each designed property.
翻译:服务机器人的可靠性和稳健性的一个重要先决条件是确保机器人在从事各种感兴趣的任务时的正确行为。 广泛测试是确保行为正确性的一种既定方法;随着学习方法融入机器人软件结构,这一点变得更加重要,因为往往无法从理论上保证在各种情况下采用这种方法。 在本文中,我们的目标是通过自动生成模拟测试情景来评价桌面操作中的机器人行为的正确性,机器人在其中使用基于属性的测试来评估其性能。特别是,各种机器人行为的主要利益特性在一种肿瘤学行动中被编码,然后在模拟环境中进行验证和验证。我们用丰田人类支持机器人(HSR)来评估我们的框架,该机器人在加泽博模拟中测试。我们表明,我们的框架可以正确和一致地识别在各种随机化桌面操纵情景中的各种失败行为,此外还能更深入地了解每个设计属性的失败类型和地点。