Deep neural networks have been well-known for their superb performance in handling various machine learning and artificial intelligence tasks. However, due to their over-parameterized black-box nature, it is often difficult to understand the prediction results of deep models. In recent years, many interpretation tools have been proposed to explain or reveal the ways that deep models make decisions. In this paper, we review this line of research and try to make a comprehensive survey. Specifically, we introduce and clarify two basic concepts-interpretations and interpretability-that people usually get confused. First of all, to address the research efforts in interpretations, we elaborate the design of several recent interpretation algorithms, from different perspectives, through proposing a new taxonomy. Then, to understand the results of interpretation, we also survey the performance metrics for evaluating interpretation algorithms. Further, we summarize the existing work in evaluating models' interpretability using "trustworthy" interpretation algorithms. Finally, we review and discuss the connections between deep models' interpretations and other factors, such as adversarial robustness and data augmentations, and we introduce several open-source libraries for interpretation algorithms and evaluation approaches.
翻译:深心神经网络因其在处理各种机器学习和人工智能任务方面的超强性能而广为人知。然而,由于其超度参数化的黑盒性质,往往难以理解深层模型的预测结果。近年来,提出了许多解释工具来解释或揭示深层模型的决策方式。在本文件中,我们审查了这一研究线并试图进行全面调查。具体地说,我们介绍并澄清了两种基本概念——解释和可解释性,人们通常会感到困惑。首先,为了解决解释方面的研究工作,我们从不同角度详细设计了最近若干解释算法,提出了新的分类法。然后,为了了解解释的结果,我们还调查了用于评价解释算法的业绩计量。此外,我们总结了目前利用“可信”解释算法评估模型可解释性的工作。最后,我们审查并讨论深模型解释与其他因素之间的联系,例如对抗性强和数据增强性,我们引入了数个用于解释算法和评估方法的开源图书馆。