Image inpainting is a restoration method that reconstructs missing image parts. However, a carefully selected mask of known pixels that yield a high quality inpainting can also act as a sparse image representation. This challenging spatial optimisation problem is essential for practical applications such as compression. So far, it has been almost exclusively adressed by model-based approaches. First attempts with neural networks seem promising, but are tailored towards specific inpainting operators or require postprocessing. To address this issue, we propose the first generative adversarial network (GAN) for spatial inpainting data optimisation. In contrast to previous approaches, it allows joint training of an inpainting generator and a corresponding mask optimisation network. With a Wasserstein distance, we ensure that our inpainting results accurately reflect the statistics of natural images. This yields significant improvements in visual quality and speed over conventional stochastic models. It also outperforms current spatial optimisation networks.
翻译:图像映射是一种重建缺失图像部件的恢复方法。 但是, 一个精心选择的已知像素的掩码, 能够产生高品质的绘画, 也可以作为一种稀有的图像表示方式。 这个具有挑战性的空间优化问题对于压缩等实际应用来说至关重要。 到目前为止, 它几乎完全被基于模型的方法所困扰。 神经网络的最初尝试似乎很有希望, 但是针对特定的绘画操作员或需要后处理的。 为了解决这个问题, 我们提议了第一个用于空间涂色数据优化的基因对抗网络( GAN ) 。 与以往的方法不同, 它允许联合培训一个油漆生成器和一个相应的遮罩优化网络。 我们用瓦瑟斯坦距离确保我们的画画结果能够准确地反映自然图像的统计。 这在视觉质量和速度方面比常规的光学模型有显著改进。 它还超越了当前的空间优化网络 。