Rain is one of the most common weather which can completely degrade the image quality and interfere with the performance of many computer vision tasks, especially under heavy rain conditions. We observe that: (i) rain is a mixture of rain streaks and rainy haze; (ii) the scene depth determines the intensity of rain streaks and the transformation into the rainy haze; (iii) most existing deraining methods are only trained on synthetic rainy images, and hence generalize poorly to the real-world scenes. Motivated by these observations, we propose a new SEMI-supervised Mixture Of rain REmoval Generative Adversarial Network (Semi-MoreGAN), which consists of four key modules: (I) a novel attentional depth prediction network to provide precise depth estimation; (ii) a context feature prediction network composed of several well-designed detailed residual blocks to produce detailed image context features; (iii) a pyramid depth-guided non-local network to effectively integrate the image context with the depth information, and produce the final rain-free images; and (iv) a comprehensive semi-supervised loss function to make the model not limited to synthetic datasets but generalize smoothly to real-world heavy rainy scenes. Extensive experiments show clear improvements of our approach over twenty representative state-of-the-arts on both synthetic and real-world rainy images.
翻译:降雨是最常见的天气之一,它能够完全降低图像质量,干扰许多计算机视觉任务的执行,特别是在大雨条件下。我们注意到:(一) 降雨是雨量和雨雾的混合体;(二) 景色深度决定雨量的强度和向雨雾的转化;(三) 大部分现有的排污方法仅受过合成雨量图像的培训,因此对真实世界的场景来说也差强人意。受这些观察的驱动,我们提议建立一个由SEMI所监督的雨水REmoval General Adversarial网络(Semi-MoreGAN),该网络由四个关键模块组成:(一) 一个新的关注深度预测网络,以提供准确的深度估计;(二) 由几个精心设计的详细的详细残留区组成的背景预测网络,以产生详细的图像背景特征特征;(三) 金字塔式的深度引导非本地网络,以有效地将图像背景与深度信息相结合,并制作最后的无雨图像;以及(四) 全面的半监测性实际损失功能,以模型为基础,在20世纪上进行真正的模拟的合成模型,不局限于。