Image inpainting aims to restore the missing regions and make the recovery results identical to the originally complete image, which is different from the common generative task emphasizing the naturalness of generated images. Nevertheless, existing works usually regard it as a pure generation problem and employ cutting-edge generative techniques to address it. The generative networks fill the main missing parts with realistic contents but usually distort the local structures. In this paper, we formulate image inpainting as a mix of two problems, i.e., predictive filtering and deep generation. Predictive filtering is good at preserving local structures and removing artifacts but falls short to complete the large missing regions. The deep generative network can fill the numerous missing pixels based on the understanding of the whole scene but hardly restores the details identical to the original ones. To make use of their respective advantages, we propose the joint predictive filtering and generative network (JPGNet) that contains three branches: predictive filtering & uncertainty network (PFUNet), deep generative network, and uncertainty-aware fusion network (UAFNet). The PFUNet can adaptively predict pixel-wise kernels for filtering-based inpainting according to the input image and output an uncertainty map. This map indicates the pixels should be processed by filtering or generative networks, which is further fed to the UAFNet for a smart combination between filtering and generative results. Note that, our method as a novel framework for the image inpainting problem can benefit any existing generation-based methods. We validate our method on three public datasets, i.e., Dunhuang, Places2, and CelebA, and demonstrate that our method can enhance three state-of-the-art generative methods (i.e., StructFlow, EdgeConnect, and RFRNet) significantly with the slightly extra time cost.
翻译:图像中的图像旨在恢复缺失区域, 使恢复结果与原始完整的图像完全相同, 这与强调生成图像自然性的常见基因化任务不同, 与强调生成图像自然性的常见基因化任务不同。 然而, 现有的基因化网络通常将它视为纯生成问题, 并且使用最先进的基因化技术来解决它。 为了利用它们各自的优点, 我们建议联合的预测过滤和基因化网络( JGGNet), 它包含三个分支: 预测过滤和不确定性时间网络( PFUNet ), 深度基因化过滤网络在保存本地结构和清除文物方面十分出色, 但要完成大量缺失的区域。 深层基因化网络网络可以填补基于对整个场景的理解而缺失的许多缺失的像素, 但几乎无法恢复与原始图像相同的细节 。 我们的智能过滤和基因化网络( JPGGNet ) 包含三个分支: 预知过滤和不确定性时间网络( PFPFUNet), 深层次化网络, 以及不确定性网络( UAFNet ) 。 PFNet 可以对智能化的种子化方法进行精确化预测, 和预变变现的图像化, 图像化方法可以显示我们的图像化方法 。