Self-supervised contrastive learning between pairs of multiple views of the same image has been shown to successfully leverage unlabeled data to produce meaningful visual representations for both natural and medical images. However, there has been limited work on determining how to select pairs for medical images, where availability of patient metadata can be leveraged to improve representations. In this work, we develop a method to select positive pairs coming from views of possibly different images through the use of patient metadata. We compare strategies for selecting positive pairs for chest X-ray interpretation including requiring them to be from the same patient, imaging study or laterality. We evaluate downstream task performance by fine-tuning the linear layer on 1% of the labeled dataset for pleural effusion classification. Our best performing positive pair selection strategy, which involves using images from the same patient from the same study across all lateralities, achieves a performance increase of 14.4% in mean AUC from the ImageNet pretrained baseline. Our controlled experiments show that the keys to improving downstream performance on disease classification are (1) using patient metadata to appropriately create positive pairs from different images with the same underlying pathologies, and (2) maximizing the number of different images used in query pairing. In addition, we explore leveraging patient metadata to select hard negative pairs for contrastive learning, but do not find improvement over baselines that do not use metadata. Our method is broadly applicable to medical image interpretation and allows flexibility for incorporating medical insights in choosing pairs for contrastive learning.
翻译:在同一图像的多个视图的对配之间自监督的对比性学习被显示成功地利用了未贴标签的数据,为自然图像和医疗图像提供有意义的直观表现。然而,在确定如何选择医疗图像配对方面,可以利用病人元数据的可用性来改善表现。在这项工作中,我们开发了一种方法,从可能不同图像的观点中选择正对,通过使用病人元数据来选择。我们比较了选择胸前X射线解释正对的策略,包括要求他们来自同一病人、成像研究或横向。我们通过将标签数据集的1%的线性图层微调来评估下游任务表现。我们的最佳执行积极的配对选择战略,就是利用同一研究的直观性能来改善病情。我们最好的执行积极的配对战略,就是利用同一研究的直观性能来利用同一研究中的同一位病人的图像的图像,并尽量利用不同的直观性能来选择不同的直观性能。我们使用不同的直观性能来进行对比。