Fully-supervised shadow removal methods achieve top restoration qualities on public datasets but still generate some shadow remnants. One of the reasons is the lack of large-scale shadow & shadow-free image pairs. Unsupervised methods can alleviate the issue but their restoration qualities are much lower than those of fully-supervised methods. In this work, we find that pretraining shadow removal networks on the image inpainting dataset can reduce the shadow remnants significantly: a naive encoder-decoder network gets competitive restoration quality w.r.t. the state-of-the-art methods via only 10% shadow & shadow-free image pairs. We further analyze the difference between networks with/without inpainting pretraining and observe that: inpainting pretraining enhances networks' capability of filling missed semantic information; shadow removal fine-tuning makes the networks know how to fill details of the shadow regions. Inspired by the above observations, we formulate shadow removal as a shadow-guided inpainting task to take advantage of the shadow removal and image inpainting. Specifically, we build a shadow-informed dynamic filtering network with two branches: the image inpainting branch takes the shadow-masked image as input while the second branch takes the shadow image as input and is to estimate dynamic kernels and offsets for the first branch to provide missing semantic information and details. The extensive experiments show that our method empowered with inpainting outperforms all state-of-the-art methods.
翻译:完全监督的阴影清除方法在公共数据集中达到了顶级恢复质量,但仍产生一些影子残余。 原因之一是缺少大型的影子和无影子图像配对。 无监督的方法可以缓解问题, 但其恢复质量比完全监督的方法要低得多。 在这项工作中, 我们发现在图像涂漆数据集上培训影子清除网络可以显著减少阴影残余: 幼稚的编码解码网络获得有竞争力的恢复质量 w.r. t。 仅通过10%的阴影和无阴影图像配对, 最先进的方法才有一定的恢复质量 。 我们进一步分析使用/ 不涂抹预培训的网络之间的差异, 并且观察: 插入前培训会提高网络填充缺失的语义信息的能力; 阴影去除微调让网络知道如何填充阴影区域的细节 。 在以上观察的启发下, 我们将阴影清除作为影子清除的导导出任务, 以利用阴影清除和图像涂抹中的图像配比 。 具体地, 我们用一个隐藏的、 隐藏的、 动态的图像过滤网络在两个分支中, 提供影化的、 显示机动的系统, 以显示机的胶变的图像转换为显示机的图。