Recent years have shown an increased development of methods for justifying the predictions of neural networks through visual explanations. These explanations usually take the form of heatmaps which assign a saliency (or relevance) value to each pixel of the input image that expresses how relevant the pixel is for the prediction of a label. Complementing this development, evaluation methods have been proposed to assess the "goodness" of such explanations. On the one hand, some of these methods rely on synthetic datasets. However, this introduces the weakness of having limited guarantees regarding their applicability on more realistic settings. On the other hand, some methods rely on metrics for objective evaluation. However the level to which some of these evaluation methods perform with respect to each other is uncertain. Taking this into account, we conduct a comprehensive study on a subset of the ImageNet-1k validation set where we evaluate a number of different commonly-used explanation methods following a set of evaluation methods. We complement our study with sanity checks on the studied evaluation methods as a means to investigate their reliability and the impact of characteristics of the explanations on the evaluation methods. Results of our study suggest that there is a lack of coherency on the grading provided by some of the considered evaluation methods. Moreover, we have identified some characteristics of the explanations, e.g. sparsity, which can have a significant effect on the performance.
翻译:近年来,通过视觉解释来证明神经网络预测的方法得到了越来越多的发展。这些解释通常采用热力图的形式,它们为输入图像的每个像素分配一个显著性(或相关性)值,该值表达该像素对标签预测的相关性。为了补充这一发展,提出了评估方法来评估这些解释的“优良性”。一方面,有些方法依赖于合成数据集。然而,这会产生在更现实的设置下应用这些方法的有限保证的弱点。另一方面,有些方法依赖于客观评估的指标。然而,一些评估方法相对于彼此的表现程度是不确定的。考虑到这一点,我们在 ImageNet-1k 验证集的一个子集上进行了全面的研究,评估了一些常用的解释方法,并按照一组评估方法进行了评估。我们将研究评估方法的可靠性和解释特性对评估方法的影响作为一种检查,结果表明一些评估方法的分级缺乏一致性。此外,我们还确定了一些解释的特性,例如稀疏性,它们对表现有显著影响。