Detecting out-of-distribution (OOD) data has become a critical component in ensuring the safe deployment of machine learning models in the real world. Existing OOD detection approaches primarily rely on the output or feature space for deriving OOD scores, while largely overlooking information from the gradient space. In this paper, we present GradNorm, a simple and effective approach for detecting OOD inputs by utilizing information extracted from the gradient space. GradNorm directly employs the vector norm of gradients, backpropagated from the KL divergence between the softmax output and a uniform probability distribution. Our key idea is that the magnitude of gradients is higher for in-distribution (ID) data than that for OOD data, making it informative for OOD detection. GradNorm demonstrates superior performance, reducing the average FPR95 by up to 16.33% compared to the previous best method.
翻译:检测分配外数据已成为确保在现实世界中安全部署机器学习模型的一个关键组成部分。现有的 OOD 检测方法主要依靠输出或特性空间来得出 OOD 分数,但基本上忽略了从梯度空间获得的信息。在本文件中,我们介绍了GradNorm,这是一个利用从梯度空间提取的信息来检测 OOD 输入的简单而有效的方法。 GradNorm 直接使用梯度的矢量规范,从软模输出与统一概率分布之间的 KL 差异中反射出来。我们的主要想法是,在分布(ID) 数据方面,梯度的大小高于OOD 数据,从而为 OOD 检测提供信息。 GradNorm 展示了优异性表现,将平均FPR95 降低到 16.33%,而前一种是最佳方法。