Fisheye lens gains increasing applications in computational photography and assisted driving because of its wide field of view (FoV). However, the fisheye image generally contains invalid black regions induced by its imaging model. In this paper, we present a FisheyeEX method that extends the FoV of the fisheye lens by outpainting the invalid regions, improving the integrity of captured scenes. Compared with the rectangle and undistorted image, there are two challenges for fisheye image outpainting: irregular painting regions and distortion synthesis. Observing the radial symmetry of the fisheye image, we first propose a polar outpainting strategy to extrapolate the coherent semantics from the center to the outside region. Such an outpainting manner considers the distribution pattern of radial distortion and the circle boundary, boosting a more reasonable completion direction. For the distortion synthesis, we propose a spiral distortion-aware perception module, in which the learning path keeps consistent with the distortion prior of the fisheye image. Subsequently, a scene revision module rearranges the generated pixels with the estimated distortion to match the fisheye image, thus extending the FoV. In the experiment, we evaluate the proposed FisheyeEX on three popular outdoor datasets: Cityscapes, BDD100k, and KITTI, and one real-world fisheye image dataset. The results demonstrate that our approach significantly outperforms the state-of-the-art methods, gaining around 27% more content beyond the original fisheye image.
翻译:鱼眼透镜在计算摄影中的应用增加,并因其视野宽广(FoV)而协助驾驶。然而,鱼眼图像一般包含由成像模型引出的无效黑区域。在本文中,我们展示了一种FisheyEX方法,通过涂抹无效区域,扩大鱼眼透镜的FoV,改善被捕获场景的完整性。与矩形和未扭曲的图像相比,鱼眼图像外涂漆有两个挑战:不规则的油漆区和扭曲合成。观察鱼眼图像的辐射对称,我们首先提出一种极偏差外涂色战略,将鱼眼图像的连贯语义从中心推至外部区域。这种偏差方式考虑鱼类眼透镜透镜的分布模式,超越了无效区域,提高了被捕获场的完整。关于扭曲的合成,我们提出了一个螺旋扭曲的扭曲感知觉感知模块,其中的学习路径与鱼眼图像的扭曲相一致。随后,一个场变模型将生成的象与鱼眼图像的估测结果相匹配,比鱼眼中心至外部区域的鱼眼图象图象图象图象图像更接近,从而展示了真正的鱼眼图层图层图象,从而展示了一条图象。