This work aims to generate realistic anatomical deformations from static patient scans. Specifically, we present a method to generate these deformations/augmentations via deep learning driven respiratory motion simulation that provides the ground truth for validating deformable image registration (DIR) algorithms and driving more accurate deep learning based DIR. We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images and predicts future breathing phases given a static CT image. The predicted respiratory patterns, represented by time-varying displacement vector fields (DVFs) at different breathing phases, are modulated through auxiliary inputs of 1D breathing traces so that a larger amplitude in the trace results in more significant predicted deformation. Stacked 3D-ConvLSTMs are used to capture the spatial-temporal respiration patterns. Training loss includes a smoothness loss in the DVF and mean-squared error between the predicted and ground truth phase images. A spatial transformer deforms the static CT with the predicted DVF to generate the predicted phase image. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. The trained RMSim was then used to augment a public DIR challenge dataset for training VoxelMorph to show the effectiveness of RMSim-generated deformation augmentation. We validated our RMSim output with both private and public benchmark datasets (healthy and cancer patients). The proposed approach can be used for validating DIR algorithms as well as for patient-specific augmentations to improve deep learning DIR algorithms. The code, pretrained models, and augmented DIR validation datasets will be released at https://github.com/nadeemlab/SeqX2Y.
翻译:这项工作旨在从静态病人扫描中产生现实的解剖变形。 具体地说, 我们展示了一种方法, 通过深层学习驱动呼吸运动模拟来产生这些变形/ 放大, 为验证变形图像登记(DIR)算法提供地面真相, 并驱动更精确的深层学习 DIR 。 我们展示了一个新的 3D Seq2Seqeq深层学习的呼吸运动模拟器(RMSim), 从 4D-CT 图像中学习, 预测未来呼吸阶段的呼吸阶段。 预测的呼吸阶段模式, 由在不同呼吸阶段的变异变流矢量矢量字段(DVF) 代表的呼吸模式(DVFs) 进行调节, 通过1D呼吸痕迹的辅助输入进行调节, 从而在跟踪结果中实现更大的振荡度, 预测变形变形的DConvLSTM 用于获取空间- 脉冲动模式。 培训损失包括DVBF 和平均直径直径图像阶段图像的变换。 空间变形变异变码, 用于预测的DVFRMRMRRM 数据变变动数据 用于预测的内变压数据变现数据, 用于预测的 RDRMRMRMRMRF 数据变现, 用于用于用于 IMRVVVVD 数据变现数据 数据 数据变换为 数据 数据 的 的变现到 的 IMRVVVT 数据 数据 数据 的变换代号。