A recent popular approach to out-of-distribution (OOD) detection is based on a self-supervised learning technique referred to as contrastive learning. There are two main variants of contrastive learning, namely instance and class discrimination, targeting features that can discriminate between different instances for the former, and different classes for the latter. In this paper, we aim to understand the effectiveness and limitation of existing contrastive learning methods for OOD detection. We approach this in 3 ways. First, we systematically study the performance difference between the instance discrimination and supervised contrastive learning variants in different OOD detection settings. Second, we study which in-distribution (ID) classes OOD data tend to be classified into. Finally, we study the spectral decay property of the different contrastive learning approaches and examine how it correlates with OOD detection performance. In scenarios where the ID and OOD datasets are sufficiently different from one another, we see that instance discrimination, in the absence of fine-tuning, is competitive with supervised approaches in OOD detection. We see that OOD samples tend to be classified into classes that have a distribution similar to the distribution of the entire dataset. Furthermore, we show that contrastive learning learns a feature space that contains singular vectors containing several directions with a high variance which can be detrimental or beneficial to OOD detection depending on the inference approach used.
翻译:最近流行的传播方法(OOD)的检测方法基于一种自我监督的学习方法,称为对比性学习; 区别性学习有两种主要变式,即实例和阶级歧视,针对不同情况区分前者的不同情况,而后者则针对不同类别。在本文件中,我们的目标是了解现有差异性学习方法在OOOD检测中的有效性和局限性。我们从三个角度来处理这个问题。首先,我们系统地研究不同OOOD检测环境中实例差别和受监督的对比性学习变量之间的性能差异。第二,我们研究在分配(ID)类OOOD数据中往往被分类为类别。最后,我们研究不同对比性学习方法的光谱衰质属性,并研究它与OOD检测性能如何相关。在ID和OD数据集彼此差异很大的情况下,我们看到,在不作微调的情况下,在OOD检测中,差异性能与受监督的方法相比是竞争性的。我们发现OOD样本往往被分类为类似于整个数据分布的类别。最后,我们研究不同对比的是不同差异性方法,我们学习了用于不同方向。