End-to-end deep neural networks (DNNs) have become the state-of-the-art (SOTA) for solving inverse problems. Despite their outstanding performance, during deployment, such networks are sensitive to minor variations in the testing pipeline and often fail to reconstruct small but important details, a feature critical in medical imaging, astronomy, or defence. Such instabilities in DNNs can be explained by the fact that they ignore the forward measurement model during deployment, and thus fail to enforce consistency between their output and the input measurements. To overcome this, we propose a framework that transforms any DNN for inverse problems into a measurement-consistent one. This is done by appending to it an implicit layer (or deep equilibrium network) designed to solve a model-based optimization problem. The implicit layer consists of a shallow learnable network that can be integrated into the end-to-end training while keeping the SOTA DNN fixed. Experiments on single-image super-resolution show that the proposed framework leads to significant improvements in reconstruction quality and robustness over the SOTA DNNs.
翻译:端到端深神经网络(DNNs)已成为解决反向问题的最先进技术(SOTA) 。尽管这些网络在部署期间表现出色,但在测试管道的细小变化中十分敏感,而且往往未能重建医疗成像、天文学或防御方面至关重要的小型但重要的细节,DNNs的这种不稳定性可以解释为它们忽视了部署期间的远端测量模型,因而无法强制实施其产出与输入测量的一致性。要克服这一点,我们建议了一个框架,将任何反向问题的DNN变成一个符合测量标准的框架。这是通过附着一个旨在解决基于模型优化问题的隐含层(或深平衡网络)来完成的。隐含层是一个可学习的浅层网络,可以纳入端到端培训,同时保持SOTA DNN。 单模超分辨率实验表明,拟议的框架可以大大改进SOTA DNes的重建质量和稳健性。</s>