Contemporary deep-learning object detection methods for autonomous driving usually assume prefixed categories of common traffic participants, such as pedestrians and cars. Most existing detectors are unable to detect uncommon objects and corner cases (e.g., a dog crossing a street), which may lead to severe accidents in some situations, making the timeline for the real-world application of reliable autonomous driving uncertain. One main reason that impedes the development of truly reliably self-driving systems is the lack of public datasets for evaluating the performance of object detectors on corner cases. Hence, we introduce a challenging dataset named CODA that exposes this critical problem of vision-based detectors. The dataset consists of 1500 carefully selected real-world driving scenes, each containing four object-level corner cases (on average), spanning more than 30 object categories. On CODA, the performance of standard object detectors trained on large-scale autonomous driving datasets significantly drops to no more than 12.8% in mAR. Moreover, we experiment with the state-of-the-art open-world object detector and find that it also fails to reliably identify the novel objects in CODA, suggesting that a robust perception system for autonomous driving is probably still far from reach. We expect our CODA dataset to facilitate further research in reliable detection for real-world autonomous driving. Our dataset will be released at https://coda-dataset.github.io.
翻译:现代自主驾驶的深层学习对象探测方法通常假定预设的通用交通参与者类别,如行人和汽车等。大多数现有探测器无法探测异常物体和角落案例(例如,一条狗跨越街道),在某些情况下可能导致严重事故,使可靠自主驾驶实际应用可靠自主驾驶实际世界的时间表不确定。阻碍开发真正可靠自我驾驶系统的一个主要原因是缺乏用于评价角落物体探测器性能的公共数据集。因此,我们引入了一个称为CODA的具有挑战性的数据集,暴露了视像探测器这一关键问题。数据集由1500个精心选择的现实世界驾驶场组成,每个场包含4个目标水平的角落案例(平均),范围超过30个物体类别。在CODA上,在大型自主驾驶数据集培训的标准物体探测器的性能显著下降至不超过12.8%。此外,我们实验了最先进的开放世界天体探测器,发现它也未能可靠地识别到CODA中的小物体。这意味着,一个可靠的自动驱动数据系统可能远达我们的CODA。我们所释放的自动驱动数据。