A Framework for the Systematic Assessment of Anomaly Detectors in Time-Sensitive Automotive Networks

Connected cars are susceptible to cyberattacks. Security and safety of future vehicles highly depend on a holistic protection of automotive components, of which the time-sensitive backbone network takes a significant role. These onboard Time-Sensitive Networks (TSNs) require monitoring for safety and -- as versatile platforms to host Network Anomaly Detection Systems (NADSs) -- for security. Still a thorough evaluation of anomaly detection methods in the context of hard real-time operations, automotive protocol stacks, and domain specific attack vectors is missing along with appropriate input datasets. In this paper, we present an assessment framework that allows for reproducible, comparable, and rapid evaluation of detection algorithms. It is based on a simulation toolchain, which contributes configurable topologies, traffic streams, anomalies, attacks, and detectors. We demonstrate the assessment of NADSs in a comprehensive in-vehicular network with its communication flows, on which we model traffic anomalies. We evaluate exemplary detection mechanisms and reveal how the detection performance is influenced by different combinations of TSN traffic flows and anomaly types. Our approach translates to other real-time Ethernet domains, such as industrial facilities, airplanes, and UAVs.