Evaluating and Improving Safety of Object Detectors in Autonomous Driving

Object detection is a key function in machine perception. Usually, their performance is evaluated based on accuracy metrics such as mean Average Precision (mAP). In this paper, we examine object detectors by their safety in the context of Autonomous Driving (AD). More concretely, we find mAP, which in turn employs the Intersection-over-Union (IoU) measure, not particularly suitable for the notion of safety in AD. Instead, we propose a novel safety metric as a more direct safety reflector, using the Intersection-over-Ground-Truth (IoGT) measure and a distance ratio between predictions and ground truths. We also formulate a safety-aware loss function that can improve an object detector and significantly reduce its unsafe predictions, compared to ordinary ones such as the SmoothL1 loss. Our experiments with open-sourced models and two datasets demonstrate the validity of our consideration and proposals.