Identifying Safety-critical Scenarios for Autonomous Vehicles via Key Features

A test scenario for an autonomous vehicle (AV) is considered safety-critical if it identifies any malfunctioning of the AV. Safety-critical test scenarios are rare under usual traffic conditions, hence simulations are often used to generate such scenarios. The representation of driving scenarios is complex, containing a multitude of static and dynamic features related to the AV, road users, such as other vehicles and pedestrians, and weather and road conditions. This makes the generation of an exhaustive test suite to identify critical test scenarios impractical, even virtually. In this paper, we present a systematic technique for the identification of significant features of test scenarios that impact their effectiveness, based on Instance Space Analysis (ISA). ISA identifies a combination of features that best differentiates safety-critical scenarios from normal driving scenarios and visualises the distribution of these features with respect to test scenario outcome. The visualisation helps identify the untested areas of the instance space where test scenarios are empirically possible, however, missing from the current test suite, which is an indicator of the quality of the test suite. We train four Machine Learning approaches to classify test scenarios as safety-critical or not. The high precision, recall, and F1-scores indicate that the proposed approach is effective in predicting the outcome of a test scenario before simulating it, thus aiding with test scenario prioritization.