Self-driving cars and trucks, autonomous vehicles (AVs), should not be accepted by regulatory bodies and the public until they have much higher confidence in their safety and reliability -- which can most practically and convincingly be achieved by testing. But existing testing methods are inadequate for checking the end-to-end behaviors of AV controllers against complex, real-world corner cases involving interactions with multiple independent agents such as pedestrians and human-driven vehicles. While test-driving AVs on streets and highways fails to capture many rare events, existing simulation-based testing methods mainly focus on simple scenarios and do not scale well for complex driving situations that require sophisticated awareness of the surroundings. To address these limitations, we propose a new fuzz testing technique, called AutoFuzz, which can leverage widely-used AV simulators' API grammars. to generate semantically and temporally valid complex driving scenarios (sequences of scenes). AutoFuzz is guided by a constrained Neural Network (NN) evolutionary search over the API grammar to generate scenarios seeking to find unique traffic violations. Evaluation of our prototype on one state-of-the-art learning-based controller and two rule-based controllers shows that AutoFuzz efficiently finds hundreds of realistic traffic violations resembling real-world crashes. Further, fine-tuning the learning-based controller with the traffic violations found by AutoFuzz successfully reduced the traffic violations found in the new version of the AV controller software.
翻译:自行驾驶的汽车和卡车、自主车辆等,不应被监管机构和公众所接受,除非它们对其安全和可靠性的信心大大提高 -- -- 最实际和令人信服地可以通过测试实现。但现有的测试方法不足以检查AV控制器的端到端行为,防止与行人和人驾驶的车辆等多个独立代理人互动的复杂、真实世界角落案件。在街头和高速公路上试驾驶AV未能捕捉到许多罕见事件的同时,现有的模拟测试方法主要侧重于简单的假设情况,而对于需要深入了解周围情况的复杂驾驶情况则不甚乐观。为了解决这些限制,我们建议采用新的模糊测试技术,称为AutoFuzz,它可以利用广泛使用的AV模拟器的API 缩写语法来控制API 的端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端到端都都都都都都到端到端到端到端到端到端到端到端到端到端都都都都都都都都都都都都都都要到端到的违反的违反的违反。