Current learning-based single image super-resolution (SISR) algorithms underperform on real data due to the deviation in the assumed degrada-tion process from that in the real-world scenario. Conventional degradation processes consider applying blur, noise, and downsampling (typicallybicubic downsampling) on high-resolution (HR) images to synthesize low-resolution (LR) counterparts. However, few works on degradation modelling have taken the physical aspects of the optical imaging system intoconsideration. In this paper, we analyze the imaging system optically andexploit the characteristics of the real-world LR-HR pairs in the spatial frequency domain. We formulate a real-world physics-inspired degradationmodel by considering bothopticsandsensordegradation; The physical degradation of an imaging system is modelled as a low-pass filter, whose cut-off frequency is dictated by the object distance, the focal length of thelens, and the pixel size of the image sensor. In particular, we propose to use a convolutional neural network (CNN) to learn the cutoff frequency of real-world degradation process. The learned network is then applied to synthesize LR images from unpaired HR images. The synthetic HR-LR image pairs are later used to train an SISR network. We evaluatethe effectiveness and generalization capability of the proposed degradation model on real-world images captured by different imaging systems. Experimental results showcase that the SISR network trained by using our synthetic data performs favorably against the network using the traditional degradation model. Moreover, our results are comparable to that obtained by the same network trained by using real-world LR-HR pairs, which are challenging to obtain in real scenes.
翻译:目前基于学习的单一图像超分辨率(SISR)算法低于真实数据,原因是假设的降解过程与现实世界情景中的情况不同。常规降解过程考虑在高分辨率(HR)图像上应用模糊、噪音和下抽样(典型的立体下抽样),以合成低分辨率(LR)对等图像。然而,关于降解模型的作品很少将光学成像系统物理方面纳入考虑。在本文中,我们通过光学分析成像系统,利用空间频域真实世界LR-HR对配对的特征。我们通过考虑光学和感官的降解,设计了一个真实世界物理学启发的降解模型;一个成像系统的物理降解模型是低传动过滤器,其截断频率取决于对象距离、焦距和图像传感器的像素模型大小。我们提议利用经过训练的SIS-RR网络,从实际降解网络的截断频率,我们开发的SLR-R-R图像网络,然后用经过训练的S-RM 模拟的图像,然后用SIS-RO-R 图像的模拟网络,然后用S-R 模拟的模拟的模拟图像, 将SIR-IRC-R 模拟图像应用到S-RO-I-I-L 模拟的模拟的模拟的模拟网络,然后用SLRV-L 模拟的模拟的模拟的模拟的模拟到SLM-I-I-I-S-S-S-SLM-V-S-S-SL-SL-SL-SL-SL-SL-L-SD-SD-SD-SD-SD-L-L-L-L-S-S-S-L-S-S-S-S-S-S-S-S-S-L-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-SD-SL-SDRVDRD-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-L-S-S-S-S-