Omnidirectional images (ODIs) have obtained lots of research interest for immersive experiences. Although ODIs require extremely high resolution to capture details of the entire scene, the resolutions of most ODIs are insufficient. Previous methods attempt to solve this issue by image super-resolution (SR) on equirectangular projection (ERP) images. However, they omit geometric properties of ERP in the degradation process, and their models can hardly generalize to real ERP images. In this paper, we propose Fisheye downsampling, which mimics the real-world imaging process and synthesizes more realistic low-resolution samples. Then we design a distortion-aware Transformer (OSRT) to modulate ERP distortions continuously and self-adaptively. Without a cumbersome process, OSRT outperforms previous methods by about 0.2dB on PSNR. Moreover, we propose a convenient data augmentation strategy, which synthesizes pseudo ERP images from plain images. This simple strategy can alleviate the over-fitting problem of large networks and significantly boost the performance of ODISR. Extensive experiments have demonstrated the state-of-the-art performance of our OSRT. Codes and models will be available at https://github.com/Fanghua-Yu/OSRT.
翻译:虽然ODI要求非常高的分辨率来捕捉整个场景的细节,但大多数ODI的分辨率是不够的。以前试图通过图像超分辨率(SR)来解决该问题的方法是:对等方形投影(ERP)图像的图像。然而,它们忽略了ERP在降解过程中的几何特性,其模型很难概括到真实的ERP图像。在本文中,我们提议Fisheye下取样,它模仿真实世界的成像过程并合成更现实的低分辨率样本。然后我们设计一个扭曲性能变异变异器(OSRT)来不断和自我调整ERP的扭曲。没有繁琐的过程,OSRT在PSNR上比以前的方法高出0.2B。此外,我们提出一个方便的数据增强战略,将普通图像中的假的ERP图像合成。这一简单战略可以减轻大型网络的过大问题,并大大提升ODISR的性能。广度实验已经展示了我们目前可用的AS/FRT的状态-ROPS/ROD号模型。