Multi-sequence MRIs can be necessary for reliable diagnosis in clinical practice due to the complimentary information within sequences. However, redundant information exists across sequences, which interferes with mining efficient representations by modern machine learning or deep learning models. To handle various clinical scenarios, we propose a sequence-to-sequence generation framework (Seq2Seq) for imaging-differentiation representation learning. In this study, not only do we propose arbitrary 3D/4D sequence generation within one model to generate any specified target sequence, but also we are able to rank the importance of each sequence based on a new metric estimating the difficulty of a sequence being generated. Furthermore, we also exploit the generation inability of the model to extract regions that contain unique information for each sequence. We conduct extensive experiments using three datasets including a toy dataset of 20,000 simulated subjects, a brain MRI dataset of 1,251 subjects, and a breast MRI dataset of 2,101 subjects, to demonstrate that (1) our proposed Seq2Seq is efficient and lightweight for complex clinical datasets and can achieve excellent image quality; (2) top-ranking sequences can be used to replace complete sequences with non-inferior performance; (3) combining MRI with our imaging-differentiation map leads to better performance in clinical tasks such as glioblastoma MGMT promoter methylation status prediction and breast cancer pathological complete response status prediction. Our code is available at https://github.com/fiy2W/mri_seq2seq.
翻译:在临床实践中,由于在序列中提供补充信息,多序列MRS对于临床实践的可靠诊断是必需的。然而,由于在序列中提供补充信息,在临床实践中可能需要多序列的可靠诊断。但是,跨序列中存在冗余信息,这干扰了现代机器学习或深层次学习模型的采矿效率表现。为了处理各种临床假设,我们提议了一个序列到序列生成框架(Seq2Seqeq),用于成像和差异代表性学习。在这项研究中,我们不仅建议在一个模型中任意生成3D/4D序列生成,以产生任何特定的目标序列,而且我们能够根据对生成序列的难度进行新的衡量来排列每个序列的重要性。此外,我们还利用该模型的生成能力无法提取包含每个序列独特信息的区域。我们利用三个数据集进行广泛的实验,包括20,000个模拟主题的玩具数据集、1,251个主题的大脑MRI数据集和2,101个主题的乳房MRI数据集,以表明(1)我们提议的Seq2Seq2Sequal2Seqequal real real realalational commatialationalationalation rematial commagial disactions commagial commatix commal commals commal commals commals commal commals commals commals commals commals commals commal 可以用来取代我们的完整成为1,可以用来取代我们的完整成算算算算算算算算作不完全。