Outdoor scene relighting is a challenging problem that requires good understanding of the scene geometry, illumination and albedo. Current techniques are completely supervised, requiring high quality synthetic renderings to train a solution. Such renderings are synthesized using priors learned from limited data. In contrast, we propose a self-supervised approach for relighting. Our approach is trained only on corpora of images collected from the internet without any user-supervision. This virtually endless source of training data allows training a general relighting solution. Our approach first decomposes an image into its albedo, geometry and illumination. A novel relighting is then produced by modifying the illumination parameters. Our solution capture shadow using a dedicated shadow prediction map, and does not rely on accurate geometry estimation. We evaluate our technique subjectively and objectively using a new dataset with ground-truth relighting. Results show the ability of our technique to produce photo-realistic and physically plausible results, that generalizes to unseen scenes.
翻译:外景照亮是一个具有挑战性的问题,需要很好地了解现场几何学、光化和反照率。 当前的技术受到完全监督, 需要高质量的合成素材来训练解决方案。 这种合成素材是利用从有限数据中学得的先期数据合成的。 相反, 我们建议了一种自我监督的点亮方法。 我们的方法仅针对从互联网上收集的、 没有任何用户监督的图像的团团体进行训练。 这种几乎无休止的培训数据源可以训练一种普遍点亮的解决方案。 我们的方法首先将图像分解成反光的反光方法, 然后通过修改光化参数来产生一种新的点亮效果。 我们的解决方案利用专门的影子预测图来捕捉阴影, 而不依赖精确的几何估计。 我们用带有地图重亮的新的数据集来主观和客观地评估我们的技术。 结果显示我们产生光现实和物理上看似的结果的能力, 从而向可见的场景。