Automated diagnosis prediction from medical images is a valuable resource to support clinical decision-making. However, such systems usually need to be trained on large amounts of annotated data, which often is scarce in the medical domain. Zero-shot methods address this challenge by allowing a flexible adaption to new settings with different clinical findings without relying on labeled data. Further, to integrate automated diagnosis in the clinical workflow, methods should be transparent and explainable, increasing medical professionals' trust and facilitating correctness verification. In this work, we introduce Xplainer, a novel framework for explainable zero-shot diagnosis in the clinical setting. Xplainer adapts the classification-by-description approach of contrastive vision-language models to the multi-label medical diagnosis task. Specifically, instead of directly predicting a diagnosis, we prompt the model to classify the existence of descriptive observations, which a radiologist would look for on an X-Ray scan, and use the descriptor probabilities to estimate the likelihood of a diagnosis. Our model is explainable by design, as the final diagnosis prediction is directly based on the prediction of the underlying descriptors. We evaluate Xplainer on two chest X-ray datasets, CheXpert and ChestX-ray14, and demonstrate its effectiveness in improving the performance and explainability of zero-shot diagnosis. Our results suggest that Xplainer provides a more detailed understanding of the decision-making process and can be a valuable tool for clinical diagnosis.
翻译:自动诊断预测医学图像是支持临床决策的宝贵资源。然而,这样的系统通常需要在注释数据大量的情况下进行训练,而在医学领域,这方面的数据往往是稀缺的。零样方法通过允许灵活适应不同临床发现的新设置而不依赖标记数据来解决这一挑战。此外,为了将自动诊断纳入临床工作流程,方法应当具有透明度和可解释性,增加医疗专业人员的信任并简化正确性验证。在本研究中,我们介绍了 Xplainer,这是一种新型的可解释的零样诊断框架,用于临床环境中的应用。Xplainer将对比视觉语言模型的描述分类方法适应于多标签医学诊断任务。具体而言,我们提示模型分类描述型观察结构在 X 光扫描中的存在,这是放射学家会寻找的内容,并使用描述符概率来估计诊断的可能性。我们的模型本质上是可解释的,因为最终的诊断预测直接基于底层描述符预测。我们在两个胸部 X 光数据集 CheXpert 和 ChestX-ray14 上对 Xplainer 进行评估,并证明了它在改进零样诊断的性能和可解释性方面的有效性。我们的结果表明,Xplainer 提供了更详细的决策过程理解,并可以成为临床诊断的一个有价值的工具。