Facial image inpainting is a challenging problem as it requires generating new pixels that include semantic information for masked key components in a face, e.g., eyes and nose. Recently, remarkable methods have been proposed in this field. Most of these approaches use encoder-decoder architectures and have different limitations such as allowing unique results for a given image and a particular mask. Alternatively, some approaches generate promising results using different masks with generator networks. However, these approaches are optimization-based and usually require quite a number of iterations. In this paper, we propose an efficient solution to the facial image painting problem using the Cyclic Reverse Generator (CRG) architecture, which provides an encoder-generator model. We use the encoder to embed a given image to the generator space and incrementally inpaint the masked regions until a plausible image is generated; a discriminator network is utilized to assess the generated images during the iterations. We empirically observed that only a few iterations are sufficient to generate realistic images with the proposed model. After the generation process, for the post processing, we utilize a Unet model that we trained specifically for this task to remedy the artifacts close to the mask boundaries. Our method allows applying sketch-based inpaintings, using variety of mask types, and producing multiple and diverse results. We qualitatively compared our method with the state-of-the-art models and observed that our method can compete with the other models in all mask types; it is particularly better in images where larger masks are utilized.
翻译:模糊图像涂色是一个具有挑战性的问题,因为它需要生成新的像素, 其中包括面部遮盖关键部件的语义信息, 例如眼睛和鼻子。 最近, 在这一领域提出了显著的方法 。 这些方法大多使用编码器解码器结构, 并有不同的局限性, 例如允许给特定图像和特定遮罩产生独特的结果 。 或者, 一些方法使用不同的发电机网络遮罩产生有希望的结果 。 然而, 这些方法基于优化, 通常需要大量循环。 在本文中, 我们建议了一种有效的方法来解决面部图像问题, 包括用于面部遮罩关键部件的语义信息, 例如: 眼睛和鼻子。 最近, 我们使用编码器将一个特定图像嵌入生成空间, 并逐渐插入遮蔽区域, 直至产生可信的图像 。 我们用一个歧视器网络网络来评估在循环过程中生成的图像 。 我们从经验上看, 只有少量的迭代方法足以生成出真实的图像 。 在生成模型后, 使用我们经过专门训练的图像模式, 我们用这个模型 使用一种特定的模型 方法 。