Unsupervised representation learning has proved to be a critical component of anomaly detection/localization in images. The challenges to learn such a representation are two-fold. Firstly, the sample size is not often large enough to learn a rich generalizable representation through conventional techniques. Secondly, while only normal samples are available at training, the learned features should be discriminative of normal and anomalous samples. Here, we propose to use the "distillation" of features at various layers of an expert network, pre-trained on ImageNet, into a simpler cloner network to tackle both issues. We detect and localize anomalies using the discrepancy between the expert and cloner networks' intermediate activation values given the input data. We show that considering multiple intermediate hints in distillation leads to better exploiting the expert's knowledge and more distinctive discrepancy compared to solely utilizing the last layer activation values. Notably, previous methods either fail in precise anomaly localization or need expensive region-based training. In contrast, with no need for any special or intensive training procedure, we incorporate interpretability algorithms in our novel framework for the localization of anomalous regions. Despite the striking contrast between some test datasets and ImageNet, we achieve competitive or significantly superior results compared to the SOTA methods on MNIST, F-MNIST, CIFAR-10, MVTecAD, Retinal-OCT, and two Medical datasets on both anomaly detection and localization.
翻译:未经监督的代表性学习已证明是图像中异常检测/本地化的一个关键组成部分。 学习这种代表性的挑战有两个方面。 首先,抽样规模往往不够大,不足以通过常规技术学习丰富的普遍代表性。 其次,虽然在培训中只有正常的样本,但学习的特征应该是正常和非异常的样本的区别性。 这里,我们提议在专家网络的不同层次上,在图像网络上预先培训,将特征“蒸馏”成一个简单的克隆网络,以解决这两个问题。 我们利用专家网络与克隆网络的中间激活值之间的差异,发现异常现象并将其本地化。 我们表明,在蒸馏中考虑多个中间提示,可以更好地利用专家的知识,而与仅仅利用最后一层的激活值相比,差异更大。 值得注意的是,以往的方法要么在精确的本地化方面失败,或者需要昂贵的区域培训。 相比之下,我们不需要任何特殊或强化的医学程序,因此,我们将可解释性算法纳入我们关于异常化区域本地化的新框架,根据输入的数据数据数据数据数据,尽管在SO-NBER数据测试中,在S-NMR图像和S-NMAR数据上,我们实现了显著的对比,在SAR- DADR图像和S-RO-DMAR数据测试数据测试方法之间进行了对比。