Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications and has thus been extensively studied, with a plethora of methods developed in the literature. However, the field currently lacks a unified, strictly formulated, and comprehensive benchmark, which often results in unfair comparisons and inconclusive results. From the problem setting perspective, OOD detection is closely related to neighboring fields including anomaly detection (AD), open set recognition (OSR), and model uncertainty, since methods developed for one domain are often applicable to each other. To help the community to improve the evaluation and advance, we build a unified, well-structured codebase called OpenOOD, which implements over 30 methods developed in relevant fields and provides a comprehensive benchmark under the recently proposed generalized OOD detection framework. With a comprehensive comparison of these methods, we are gratified that the field has progressed significantly over the past few years, where both preprocessing methods and the orthogonal post-hoc methods show strong potential.
翻译:分配外探测对于安全关键机器学习应用至关重要,因此已经进行了广泛研究,文献中发展了大量方法,然而,实地目前缺乏统一、严格制定和全面的基准,往往导致不公平的比较和无结果的结果。从问题确定的角度来说,OOD探测与周边领域密切相关,包括异常探测(AD)、开放确认(OSR)和模型不确定性,因为为一个领域开发的方法往往相互适用。为了帮助社区改进评价和推进,我们建立了一个统一、结构完善的代码库,称为OpenOOOD, 实施相关领域开发的30多种方法,并在最近提出的通用OOOD探测框架下提供了全面基准。我们感到满意的是,在过去几年里,实地取得了很大进展,在这些地区,预处理方法和或直角后热方法都显示出巨大的潜力。