Detecting out-of-distribution examples is important for safety-critical machine learning applications such as detecting novel biological phenomena and self-driving cars. However, existing research mainly focuses on simple small-scale settings. To set the stage for more realistic out-of-distribution detection, we depart from small-scale settings and explore large-scale multiclass and multi-label settings with high-resolution images and thousands of classes. To make future work in real-world settings possible, we create new benchmarks for three large-scale settings. To test ImageNet multiclass anomaly detectors, we introduce the Species dataset containing over 700,000 images and over a thousand anomalous species. We leverage ImageNet-21K to evaluate PASCAL VOC and COCO multilabel anomaly detectors. Third, we introduce a new benchmark for anomaly segmentation by introducing a segmentation benchmark with road anomalies. We conduct extensive experiments in these more realistic settings for out-of-distribution detection and find that a surprisingly simple detector based on the maximum logit outperforms prior methods in all the large-scale multi-class, multi-label, and segmentation tasks, establishing a simple new baseline for future work.
翻译:检测分配外的范例对于安全关键机器学习应用非常重要,例如探测新生物现象和自行驾驶汽车。然而,现有的研究主要侧重于简单的小型环境。为为更现实的分发外的检测奠定基础,我们从小规模的设置出发,探索具有高分辨率图像和数千个等级的大型多级和多标签设置。为了使在现实环境中的未来工作成为可能,我们为三个大型环境创建了新的基准。为测试图像网多级异常探测器,我们引入了包含70多万图像和一千多个异常物种的物种的物种数据集。我们利用图像网-21K来评估PASAL VOC和COCO多标签异常探测器。第三,我们引入了一条带有道路异常的分层基准,从而引入了异常分解的新基准。我们在这些更现实的环境下进行广泛的实验,以分配外的检测为目的,并发现基于最大记录层的简单探测器在所有大型多级、多标签和分块任务中都符合先前的方法,为未来工作设定了一个简单的新基准。