Saliency Maps (SMs) have been extensively used to interpret deep learning models decision by highlighting the features deemed relevant by the model. They are used on highly nonlinear problems, where linear feature selection (FS) methods fail at highlighting relevant explanatory variables. However, the reliability of gradient-based feature attribution methods such as SM has mostly been only qualitatively (visually) assessed, and quantitative benchmarks are currently missing, partially due to the lack of a definite ground truth on image data. Concerned about the apophenic biases introduced by visual assessment of these methods, in this paper we propose a synthetic quantitative benchmark for Neural Networks (NNs) interpretation methods. For this purpose, we built synthetic datasets with nonlinearly separable classes and increasing number of decoy (random) features, illustrating the challenge of FS in high-dimensional settings. We also compare these methods to conventional approaches such as mRMR or Random Forests. Our results show that our simple synthetic datasets are sufficient to challenge most of the benchmarked methods. TreeShap, mRMR and LassoNet are the best performing FS methods. We also show that, when quantifying the relevance of a few non linearly-entangled predictive features diluted in a large number of irrelevant noisy variables, neural network-based FS and interpretation methods are still far from being reliable.
翻译:瞩点图(SM)已被广泛用于通过突出模型认为相关的特征来解释深度学习模型决策。它们在高度非线性的问题上被使用,在这些问题中线性特征选择(FS)方法在突出相关的解释变量方面失败。然而,基于梯度的特征归因方法(如SM)的可靠性主要只是进行了定性(可视化)评估,目前缺乏定量基准测试,部分原因是在图像数据上缺乏明确的基本真相。由于对这些方法的视觉评估引入的幻觉偏见的关注,本文提出了一种针对神经网络(NN)解释方法的综合量化基准测试。为此,我们构建了非线性可分类的合成数据集和增加随机特征数量,以说明在高维度设置中FS的挑战。我们还将这些方法与传统方法(例如mRMR或随机森林)进行了比较。我们的结果表明,我们简单的合成数据集足以挑战大多数基准测试方法。 TreeShap,mRMR和LassoNet是表现最佳的FS方法。我们还表明,当量化某些分散在大量无关噪声变量中的几个非线性纠缠的预测特征的相关性时,基于神经网络的FS和解释方法仍然远未可靠。