Machine learning (ML) models are becoming increasingly common in the atmospheric science community with a wide range of applications. To enable users to understand what an ML model has learned, ML explainability has become a field of active research. In Part I of this two-part study, we described several explainability methods and demonstrated that feature rankings from different methods can substantially disagree with each other. It is unclear, though, whether the disagreement is overinflated due to some methods being less faithful in assigning importance. Herein, "faithfulness" or "fidelity" refer to the correspondence between the assigned feature importance and the contribution of the feature to model performance. In the present study, we evaluate the faithfulness of feature ranking methods using multiple methods. Given the sensitivity of explanation methods to feature correlations, we also quantify how much explainability faithfulness improves after correlated features are limited. Before dimensionality reduction, the feature relevance methods [e.g., SHAP, LIME, ALE variance, and logistic regression (LR) coefficients] were generally more faithful than the permutation importance methods due to the negative impact of correlated features. Once correlated features were reduced, traditional permutation importance became the most faithful method. In addition, the ranking uncertainty (i.e., the spread in rank assigned to a feature by the different ranking methods) was reduced by a factor of 2-10, and excluding less faithful feature ranking methods reduces it further. This study is one of the first to quantify the improvement in explainability from limiting correlated features and knowing the relative fidelity of different explainability methods.
翻译:在大气科学界中,机器学习模式越来越普遍,应用范围很广。为了使用户能够理解ML模型学到了什么,ML解释性已成为积极研究的领域。在这项两部分研究的第一部分,我们介绍了几种解释性方法,并表明不同方法的特征排名可能彼此大相径庭。但尚不清楚的是,由于某些方法不太忠实地赋予重要性,这种差异是否过大了。这里,“忠诚”或“忠诚”是指指定特征重要性与特征对模型性能的贡献之间的对应性。在本研究中,我们用多种方法评估特征排名方法的准确性。鉴于解释方法对特征相关性的敏感性,我们还介绍了几种解释性方法,并表明不同方法之间的差异性能差别很大。在尺寸下降之前,特征相关性方法[例如,SHAP、LME、ALE差异和物流回归系数]一般比调异的重要性方法更准确性,因为相关特性的负面影响。在本项研究中,对特征评级方法的准确性评估方法的准确性性性性性进行了评估。由于解释性化方法的敏感性降低,因此,传统的等级的稳定性变得不那么重要。