Neural networks are highly effective tools for image reconstruction problems such as denoising and compressive sensing. To date, neural networks for image reconstruction are almost exclusively convolutional. The most popular architecture is the U-Net, a convolutional network with a multi-resolution architecture. In this work, we show that a simple network based on the multi-layer perceptron (MLP)-mixer enables state-of-the art image reconstruction performance without convolutions and without a multi-resolution architecture, provided that the training set and the size of the network are moderately large. Similar to the original MLP-mixer, the image-to-image MLP-mixer is based exclusively on MLPs operating on linearly-transformed image patches. Contrary to the original MLP-mixer, we incorporate structure by retaining the relative positions of the image patches. This imposes an inductive bias towards natural images which enables the image-to-image MLP-mixer to learn to denoise images based on fewer examples than the original MLP-mixer. Moreover, the image-to-image MLP-mixer requires fewer parameters to achieve the same denoising performance than the U-Net and its parameters scale linearly in the image resolution instead of quadratically as for the original MLP-mixer. If trained on a moderate amount of examples for denoising, the image-to-image MLP-mixer outperforms the U-Net by a slight margin. It also outperforms the vision transformer tailored for image reconstruction and classical un-trained methods such as BM3D, making it a very effective tool for image reconstruction problems.
翻译:神经网络是图像重建问题非常有效的工具, 如拆解和压缩感测。 时至今日, 用于图像重建的神经网络几乎完全是卷土重来。 最受欢迎的结构是 U- Net, 是一个具有多分辨率结构的连锁网络。 在此工作中, 我们显示基于多层光谱( MLP) 混合器的简单网络可以使艺术图像重建的状态不发生变化, 也没有多分辨率结构, 只要培训组和网络规模小于中等分辨率结构。 类似于原始 MLP- 混集器, 图像到图像的图像到图像的映像器完全基于 MLP 运行, 与原始的 MLP- 图像的原始变异性模型相比, 我们通过保留图像的相对位置来整合结构。 这给自然图像带来了一种含色的偏向性偏向性, 它的图像到图像到模拟流压的中位值, MLP- 图像的原始变异性模型比原始图像的原始图像要小, 其图像到图像的变异的图像到图像的变异性图像的图像的变异性, 的图像到这样的变异的图像的变的图像, 也需要的图像到这样的图像的变异变的图像的图像的图像到比等的图像的变的图像的图像的图像的变的变的图像的变的图像的图像的图像的图像的图像的图像的变变异性, 。