Autonomous driving systems (ADSs) must be tested thoroughly before they can be deployed in autonomous vehicles. High-fidelity simulators allow them to be tested against diverse scenarios, including those that are difficult to recreate in real-world testing grounds. While previous approaches have shown that test cases can be generated automatically, they tend to focus on weak oracles (e.g. reaching the destination without collisions) without assessing whether the journey itself was undertaken safely and satisfied the law. In this work, we propose LawBreaker, an automated framework for testing ADSs against real-world traffic laws, which is designed to be compatible with different scenario description languages. LawBreaker provides a rich driver-oriented specification language for describing traffic laws, and a fuzzing engine that searches for different ways of violating them by maximising specification coverage. To evaluate our approach, we implemented it for Apollo+LGSVL and specified the traffic laws of China. LawBreaker was able to find 14 violations of these laws, including 173 test cases that caused accidents.
翻译:自动驾驶系统(ADS)必须经过彻底测试,然后才能在自主车辆中部署。高忠诚模拟器允许他们根据不同的假设情况进行测试,包括难以在现实测试场再创造的假设情况。以前的方法表明,测试案例可以自动生成,但往往侧重于薄弱的甲骨文(例如,在没有碰撞的情况下到达目的地),而没有评估旅程本身是否安全进行并符合法律要求。在这项工作中,我们提议了LawBreaker(LawBreaker)(LawBreaker)(LawBreaker)(LawBreaker)(LawBreaker),这是一个用来根据现实世界交通法测试ADS的自动框架,这个框架的设计与不同的情景描述语言兼容。LawBreaker(LawBreaker)提供了丰富的面向司机的规格语言,用于描述交通法的描述,以及一个通过最大化规格覆盖范围寻找不同违反方式的模糊引擎。为了评估我们的方法,我们为阿波波罗+LGSVL制定了交通法,并规定了中国的交通法。Legaker(LBreaker)能够发现14项违反这些法律的行为,包括173个造成事故的测试案例。