Thorough testing of safety-critical autonomous systems, such as self-driving cars, autonomous robots, and drones, is essential for detecting potential failures before deployment. One crucial testing stage is model-in-the-loop testing, where the system model is evaluated by executing various scenarios in a simulator. However, the search space of possible parameters defining these test scenarios is vast, and simulating all combinations is computationally infeasible. To address this challenge, we introduce AmbieGen, a search-based test case generation framework for autonomous systems. AmbieGen uses evolutionary search to identify the most critical scenarios for a given system, and has a modular architecture that allows for the addition of new systems under test, algorithms, and search operators. Currently, AmbieGen supports test case generation for autonomous robots and autonomous car lane keeping assist systems. In this paper, we provide a high-level overview of the framework's architecture and demonstrate its practical use cases.
翻译:安全关键自主系统(如自行驾驶汽车、自主机器人和无人驾驶飞机)的彻底测试对于在部署前发现潜在故障至关重要。一个关键的测试阶段是模拟测试,通过模拟器执行各种假设情况来评估系统模型。然而,确定这些测试情景的可能参数的搜索空间非常广阔,模拟所有组合是计算上不可行的。为了应对这一挑战,我们引入了AmbieGen,这是自主系统搜索测试案例生成框架。AmbieGen利用进化搜索来确定特定系统最关键的情景,并拥有模块化架构,允许在测试、算法和搜索操作器中添加新系统。目前,AmbieGen支持自动机器人和自动车道维护辅助系统的测试案例生成。在本文中,我们提供了框架结构的高层次概览,并展示了其实际使用案例。