Existing learning-based image inpainting methods are still in challenge when facing complex semantic environments and diverse hole patterns. The prior information learned from the large scale training data is still insufficient for these situations. Reference images captured covering the same scenes share similar texture and structure priors with the corrupted images, which offers new prospects for the image inpainting tasks. Inspired by this, we first build a benchmark dataset containing 10K pairs of input and reference images for reference-guided inpainting. Then we adopt an encoder-decoder structure to separately infer the texture and structure features of the input image considering their pattern discrepancy of texture and structure during inpainting. A feature alignment module is further designed to refine these features of the input image with the guidance of a reference image. Both quantitative and qualitative evaluations demonstrate the superiority of our method over the state-of-the-art methods in terms of completing complex holes.
翻译:在面临复杂的语义环境和不同的洞状模式时,现有的基于学习的图像油漆方法仍面临挑战。以前从大规模培训数据中获取的信息对于这些情况来说仍然不够。从相同场景中采集的参考图像与腐蚀图像有着相似的纹理和结构前缀,这为图像油漆任务提供了新的前景。受此启发,我们首先建立一个基准数据集,其中包括10K对投入和参考图像,供参考制导油漆使用。然后我们采用一种编码解码结构,分别推断输入图像的纹理和结构特征,考虑到它们在油漆过程中的纹理和结构差异。还设计了一个特征调整模块,用参考图像的指导来完善输入图像的这些特征。定量和定性评估都表明我们的方法优于完成复杂孔方面的最新方法。