We propose neural network layers that explicitly combine frequency and image feature representations and show that they can be used as a versatile building block for reconstruction from frequency space data. Our work is motivated by the challenges arising in MRI acquisition where the signal is a corrupted Fourier transform of the desired image. The proposed joint learning schemes enable both correction of artifacts native to the frequency space and manipulation of image space representations to reconstruct coherent image structures at every layer of the network. This is in contrast to most current deep learning approaches for image reconstruction that treat frequency and image space features separately and often operate exclusively in one of the two spaces. We demonstrate the advantages of joint convolutional learning for a variety of tasks, including motion correction, denoising, reconstruction from undersampled acquisitions, and combined undersampling and motion correction on simulated and real world multicoil MRI data. The joint models produce consistently high quality output images across all tasks and datasets. When integrated into a state of the art unrolled optimization network with physics-inspired data consistency constraints for undersampled reconstruction, the proposed architectures significantly improve the optimization landscape, which yields an order of magnitude reduction of training time. This result suggests that joint representations are particularly well suited for MRI signals in deep learning networks. Our code and pretrained models are publicly available at https://github.com/nalinimsingh/interlacer.
翻译:我们提议建立神经网络层,明确结合频率和图像特征的显示,并表明它们可以用作利用频率空间数据重建的多功能建筑块。我们的工作动力来自MRI获取过程中出现的挑战,因为该信号是腐败的Fourier对理想图像的变换。拟议的联合学习计划既能将原生文物校正到频率空间,又能操纵图像空间显示在网络的每个层次重建连贯的图像结构。这与目前大多数深刻的图像重建学习方法形成鲜明对照,这些方法分别处理频率和图像空间特征,并且往往在两个空间中的一个空间中专门运作。我们展示了联合革命性学习的好处,以完成各种任务,包括运动校正、淡化、从未充分抽样的购置中重建,以及模拟和真实世界多层MRI数据的综合抽样和运动校正。这些联合模型在各种任务和数据集中始终产生高质量的产出图像。当与艺术无节制优化网络的状态相结合时,以物理激发的数据一致性限制在两个空间中的一个空间中运作。我们提议的架构大大改进了优化的景观,从而产生了一种在深度的深度的模版图示。在深度的模型中生成中,这是我们学习前的模模模模。我们学习前的模型的模型中的结果。