The massive amount of electronic health records (EHR) has created enormous potential in improving healthcare. Clinical codes (structured data) and clinical narratives (unstructured data) are two important textual modalities in EHR. Clinical codes convey diagnostic and treatment information during the hospital, and clinical notes carry narratives of clinical providers for patient encounters. They do not exist in isolation and can complement each other in most real-life clinical scenarios. However, most existing EHR-oriented studies either focus on a particular modality or integrate data from different modalities in a straightforward manner, which ignores the intrinsic interactions between them. To address these issues, we proposed a Medical Multimodal Pre-trained Language Model, named MedM-PLM, to learn enhanced EHR representations over structured and unstructured data. In MedM-PLM, two Transformer-based neural network components are firstly adopted to learn representative characteristics from each modality. A cross-modal module is then introduced to model their interactions. We pre-trained MedM-PLM on the MIMIC-III dataset and verified the effectiveness of the model on three downstream clinical tasks, i.e., medication recommendation, 30-day readmission prediction and ICD coding. Extensive experiments demonstrate the power of MedM-PLM compared with state-of-the-art methods. Further analyses and visualizations show the robustness of our model, which could potentially provide more comprehensive interpretations for clinical decision-making.
翻译:大量电子健康记录(EHR)在改善医疗保健方面产生了巨大的潜力。临床代码(结构化数据)和临床叙述(非结构化数据)是EHR的两个重要文本模式。临床代码在医院期间传递诊断和治疗信息,临床记录包含临床提供者为患者所遭遇的描述,它们并非孤立存在,在大多数真实的临床假设中可以互为补充。然而,大多数现有的面向电子健康记录(EHR)的研究要么侧重于特定模式,要么以直接的方式整合不同模式的数据,忽视它们之间的内在互动。为解决这些问题,我们提议了一个医学多式预培训语言模型(MedM-PLM),以学习在结构化和非结构化数据方面的强化EHR表述。在MEM-PM中,两个基于变异器的神经网络组成部分首先用于学习每种模式的代表性特征。然后可以引入一个跨模式模块来模拟它们的互动。我们在MIMIC-III数据集上经过预先培训的M-PLM-PM数据,并核实了模型在三个下游临床任务上的有效性,即MMM-PLM(M-M-M-C-M-M-M-C-C-C-C-C-Provic-Provic-Provil-Provil-Provil-s-Provil-s-view-s-I-I-I-I-I)的更进一步展示、Misl-Proview-s-s-s-s-s-s-s-s-vial-s-s-vial-s-s-vical-viewal-viewal-s-s-s-s-vical-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-sl-sl-s-s-sl-slvical-sal-slvical-sisal-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-s-sl-s-s-s-s-s-s-s-s-s-