Deep neural networks have shown significant promise in comprehending complex visual signals, delivering performance on par or even superior to that of human experts. However, these models often lack a mechanism for interpreting their predictions, and in some cases, particularly when the sample size is small, existing deep learning solutions tend to capture spurious correlations that compromise model generalizability on unseen inputs. In this work, we propose a contrastive causal representation learning strategy that leverages proactive interventions to identify causally-relevant image features, called Proactive Pseudo-Intervention (PPI). This approach is complemented with a causal salience map visualization module, i.e., Weight Back Propagation (WBP), that identifies important pixels in the raw input image, which greatly facilitates the interpretability of predictions. To validate its utility, our model is benchmarked extensively on both standard natural images and challenging medical image datasets. We show this new contrastive causal representation learning model consistently improves model performance relative to competing solutions, particularly for out-of-domain predictions or when dealing with data integration from heterogeneous sources. Further, our causal saliency maps are more succinct and meaningful relative to their non-causal counterparts.
翻译:深心神经网络在理解复杂的视觉信号、以比人类专家高的或甚至优于人类专家的眼光提供表现方面显示了巨大的希望,然而,这些模型往往缺乏解释其预测的机制,在某些情况下,特别是当样本规模小时,现有深层的学习解决方案往往会捕捉虚假的关联性,而这种关联性会损害模型对无形投入的通用性。在这项工作中,我们提出了一个反差因果代表学习战略,利用积极性干预措施,确定与因果相关的图像特征,称为Proactive Psedo-Intervention(PPPI)。这一方法得到了一个因果显著的地图可视化模块的补充,即WEight Back Propagation(WBPPPP),该模块识别了原始输入图像中的重要像素,大大便利了预测的可解释性。为了验证其效用,我们的模型广泛以标准的自然图像和具有挑战性的医学图像数据集为基准。我们展示了这个新的反差因果代表学习模式,不断改善模型的性与竞争性解决方案的相对性,特别是外部预测或处理不同来源的数据整合时,我们的因果显著的地图与非对应性比较。