State-of-the-art machine learning models often learn spurious correlations embedded in the training data. This poses risks when deploying these models for high-stake decision-making, such as in medical applications like skin cancer detection. To tackle this problem, we propose Reveal to Revise (R2R), a framework entailing the entire eXplainable Artificial Intelligence (XAI) life cycle, enabling practitioners to iteratively identify, mitigate, and (re-)evaluate spurious model behavior with a minimal amount of human interaction. In the first step (1), R2R reveals model weaknesses by finding outliers in attributions or through inspection of latent concepts learned by the model. Secondly (2), the responsible artifacts are detected and spatially localized in the input data, which is then leveraged to (3) revise the model behavior. Concretely, we apply the methods of RRR, CDEP and ClArC for model correction, and (4) (re-)evaluate the model's performance and remaining sensitivity towards the artifact. Using two medical benchmark datasets for Melanoma detection and bone age estimation, we apply our R2R framework to VGG, ResNet and EfficientNet architectures and thereby reveal and correct real dataset-intrinsic artifacts, as well as synthetic variants in a controlled setting. Completing the XAI life cycle, we demonstrate multiple R2R iterations to mitigate different biases. Code is available on https://github.com/maxdreyer/Reveal2Revise.
翻译:最先进的机器学习模型经常学习嵌入在训练数据中的错误相关性。当将这些模型部署用于高风险决策时(例如在皮肤癌检测等医疗应用中),这就存在风险。为解决这个问题,我们提出了Reveal to Revise (R2R)框架,涵盖了整个可解释人工智能(XAI)生命周期,使从业者能够通过最少的人机交互迭代地识别、减轻和(重新)评估虚假的模型行为。在第一步(1)中,R2R通过查找归因中的异常值或检查模型学习的潜在概念来揭示模型的弱点。其次(2),发现对模型行为有影响的因素,并在输入数据中进行空间定位。然后(3)修正模型行为。具体而言,我们应用RRR,CDEP和ClArC方法进行模型修正,并(4)(重新)评估模型的性能和对因素的剩余敏感性。使用Melanoma检测和骨龄估计的两个医学基准数据集,我们将R2R框架应用于VGG、ResNet和EfficientNet架构,从而揭示并纠正实际数据集固有的因素,以及在受控环境中的合成变体。完成XAI生命周期,我们演示了多个R2R迭代,以减轻不同的偏差。代码可在https://github.com/maxdreyer/ Reveal2Revise上获得。