论文推荐 | 很可能出现在下一代PS中的深度门卷积图像补全技术

2018 年 10 月 30 日 AI研习社

AI 研习社论文推荐版块已经正式上线，欢迎大家前往社区论文板块查阅喔~

ai.yanxishe.com/page/paper

（戳文末阅读原文直接进）

Free-Form Image Inpainting with Gated Convolution

（很可能出现在下一代PS中的深度门卷积图像补全技术）

Yu Jiahui /Lin Zhe /Yang Jimei /Shen Xiaohui /Lu Xin /Huang Thomas S.

推荐原因

自拍照里有个痘痘想去掉，旅游照里乱入的路人想去掉，这种时候我们往往束手无策，只能求助好友里的PS大神。说到底这实际上是一种图像内容填充任务——选出图像中不需要的内容所在的区域，然后根据照片中周边的物体对这个区域进行填充；如果填充出了好的效果，自然就好像选出的那些内容“本来就不存在”一样了。

PhotoShop 的出品方 Adobe 公司自然知道这种任务是用户的核心需求之一，自己也在这方面做着研发工作。没有深度学习的时候，自动方法总是差强人意，有深度学习之后大可以追求更高的目标。这篇UIUC和Adobe合作的论文就展现了他们在这方面的最新成果——就像我们预想的那样，选出（用颜色遮蔽）图像中任意大小、任意形状的区域，算法就可以自动进行填充。

项目主页参见：http://jiahuiyu.com/deepfill2/

演示视频搬运参见：https://weibo.com/1402400261/Gl7cpkDWZ

来自AI研习社用户@杨晓凡的推荐

摘要

We present a novel deep learning based image inpainting system to complete images with free-form masks and inputs. The system is based on gated convolutions learned from millions of images without additional labelling efforts. The proposed gated convolution solves the issue of vanilla convolution that treats all input pixels as valid ones, generalizes partial convolution by providing a learnable dynamic feature selection mechanism for each channel at each spatial location across all layers. Moreover, as free-form masks may appear anywhere in images with any shapes, global and local GANs designed for a single rectangular mask are not suitable. To this end, we also present a novel GAN loss, named SN-PatchGAN, by applying spectral-normalized discriminators on dense image patches. It is simple in formulation, fast and stable in training. Results on automatic image inpainting and user-guided extension demonstrate that our system generates higher-quality and more flexible results than previous methods. We show that our system helps users quickly remove distracting objects, modify image layouts, clear watermarks, edit faces and interactively create novel objects in images. Furthermore, visualization of learned feature representations reveals the effectiveness of gated convolution and provides an interpretation of how the proposed neural network fills in missing regions. More high-resolution results and video materials are available at http://jiahuiyu.com/deepfill2

论文查阅地址（扫描二维码直达）：

http://www.gair.link/page/paperDetail/27