Noise synthesis is a challenging low-level vision task aiming to generate realistic noise given a clean image along with the camera settings. To this end, we propose an effective generative model which utilizes clean features as guidance followed by noise injections into the network. Specifically, our generator follows a UNet-like structure with skip connections but without downsampling and upsampling layers. Firstly, we extract deep features from a clean image as the guidance and concatenate a Gaussian noise map to the transition point between the encoder and decoder as the noise source. Secondly, we propose noise synthesis blocks in the decoder in each of which we inject Gaussian noise to model the noise characteristics. Thirdly, we propose to utilize an additional Style Loss and demonstrate that this allows better noise characteristics supervision in the generator. Through a number of new experiments, we evaluate the temporal variance and the spatial correlation of the generated noise which we hope can provide meaningful insights for future works. Finally, we show that our proposed approach outperforms existing methods for synthesizing camera noise.
翻译:噪声综合是一项具有挑战性的低级视觉任务,旨在生成与相机设置相对应的真实噪声。为此,我们提出了一种有效的生成模型,该模型利用清晰图像特征作为向导,并在网络中注入噪声。具体而言,我们的生成器采用UNet-like结构,带有跳跃连接,但不带有下采样和上采样层。首先,我们从清晰图像中提取深层特征作为向导,并在编码器和解码器之间的转换点连接一个高斯噪声映射作为噪声源。其次,在每个解码器中,我们提出噪声综合块,其中注入高斯噪声以对噪声特征进行建模。第三,我们提出使用额外的样式损失,并证明这可以使产生的噪声特征监督更加准确。通过一些新的实验,我们评估了生成噪声的时间方差和空间相关性,希望能为未来的工作提供有意义的见解。最后,我们展示了我们提出的方法优于现有的相机噪声合成方法。