Exploiting spatial-angular correlation is crucial to light field (LF) image super-resolution (SR), but is highly challenging due to its non-local property caused by the disparities among LF images. Although many deep neural networks (DNNs) have been developed for LF image SR and achieved continuously improved performance, existing methods cannot well leverage the long-range spatial-angular correlation and thus suffer a significant performance drop when handling scenes with large disparity variations. In this paper, we propose a simple yet effective method to learn the non-local spatial-angular correlation for LF image SR. In our method, we adopt the epipolar plane image (EPI) representation to project the 4D spatial-angular correlation onto multiple 2D EPI planes, and then develop a Transformer network with repetitive self-attention operations to learn the spatial-angular correlation by modeling the dependencies between each pair of EPI pixels. Our method can fully incorporate the information from all angular views while achieving a global receptive field along the epipolar line. We conduct extensive experiments with insightful visualizations to validate the effectiveness of our method. Comparative results on five public datasets show that our method not only achieves state-of-the-art SR performance, but also performs robust to disparity variations. Code is publicly available at https://github.com/ZhengyuLiang24/EPIT.
翻译:探索空间- 角关系对于光场(LF) 图像超分辨率(SR) 至关重要,但因其非本地性质,LF图像之间差异导致的非本地性质而具有高度挑战性。虽然许多深神经网络(DNNs)是为LF图像SR开发的,并不断提高性能,但现有方法无法很好地利用长距离空间- 角关系,因此在处理差异很大的场景时,其性能显著下降。在本文件中,我们提出了一个简单而有效的方法,用于学习LF图像SR(SR)的非本地空间- 角关系。在我们的方法中,我们采用上层平面图像(EPI)的表示方式,将4D空间- 角关系投射到多维2D EPI 平面上,然后开发一个具有重复性自我注意操作的变换器网络,以学习空间- 角关系,通过模拟每对 EPI 平方平方位之间的依赖性能。我们的方法可以充分纳入所有角观中的信息,同时实现全球可接受的场景线。我们进行广泛的可视化实验,我们进行广泛的视觉- EPEP/ 平面平面平面平面平面图像(EPR- ) 的图,我们只能在公开性变化方法上进行五度(S- 度) 度(S- 度) 度(SDRVDR) 度(S- R) 度(S- L) 度(S- 度) 度(S- 度) 度(S- 度) 度) 度(S) 度(SB) 度(S- 度) 度(S- sq) 度(S- squor) 度(S- ) ) 度(S- squal- sq) ) laut- sq) etal- squal- squal) roual) asetal- sal) yl) yl) view) 上进行对比测结果。