Deep anomaly detection methods learn representations that separate between normal and anomalous images. Although self-supervised representation learning is commonly used, small dataset sizes limit its effectiveness. It was previously shown that utilizing external, generic datasets (e.g. ImageNet classification) can significantly improve anomaly detection performance. One approach is outlier exposure, which fails when the external datasets do not resemble the anomalies. We take the approach of transferring representations pre-trained on external datasets for anomaly detection. Anomaly detection performance can be significantly improved by fine-tuning the pre-trained representations on the normal training images. In this paper, we first demonstrate and analyze that contrastive learning, the most popular self-supervised learning paradigm cannot be naively applied to pre-trained features. The reason is that pre-trained feature initialization causes poor conditioning for standard contrastive objectives, resulting in bad optimization dynamics. Based on our analysis, we provide a modified contrastive objective, the Mean-Shifted Contrastive Loss. Our method is highly effective and achieves a new state-of-the-art anomaly detection performance including $98.6\%$ ROC-AUC on the CIFAR-10 dataset.
翻译:深度异常探测方法可以区分正常和异常图像。虽然通常使用自我监督的代表学习方法,但小规模的数据集大小限制了其有效性。以前曾表明,利用外部通用数据集(如图像网络分类)可以大大改进异常探测性能。一种方法是外部暴露,如果外部数据集与异常情况不相像,则外部暴露就失败了。我们采取在外部数据集上预先培训以探测异常情况的方法。通过微调培训前的正常培训图像演示方法,异常探测性能可以大大改进。我们首先展示和分析这种对比性学习,最受欢迎的自我监督学习模式不能天真地适用于预先培训的特征。原因是,预先培训前的特征初始化导致标准对比目标的不适应性,导致不优化动态。根据我们的分析,我们提供了一个经过修改的对比性目标,即平均对比性损失。我们的方法非常有效,在CIFAR-10数据上实现了一种新的状态异常检测性能,包括98.6美元ROC-AUC。