Out of distribution (OOD) detection is a crucial part of making machine learning systems robust. The ImageNet-O dataset is an important tool in testing the robustness of ImageNet trained deep neural networks that are widely used across a variety of systems and applications. We aim to perform a comparative analysis of OOD detection methods on ImageNet-O, a first of its kind dataset with a label distribution different than that of ImageNet, that has been created to aid research in OOD detection for ImageNet models. As this dataset is fairly new, we aim to provide a comprehensive benchmarking of some of the current state of the art OOD detection methods on this novel dataset. This benchmarking covers a variety of model architectures, settings where we haves prior access to the OOD data versus when we don't, predictive score based approaches, deep generative approaches to OOD detection, and more.
翻译:图像网络- O 数据集是测试图像网络所训练的深神经网络是否可靠的一个重要工具,这些网络在各种系统和应用程序中广泛使用。 我们的目标是对图像网络OD探测方法进行比较分析。 图像网络OD探测方法是其同类数据集中的第一个,其标签分布不同于图像网络。 创建这种数据集是为了协助为图像网络模型进行OOD检测的研究。 由于这个数据集相当新,我们的目标是为这个新数据集提供一些最新水平的高级OOOD探测方法的全面基准。 这个基准包括各种模型结构、我们事先能够访问OOD数据的环境,以及当我们不使用OOD数据时,预测性分数方法,对OOD检测的深度基因化方法等等。