Compared to other severe weather image restoration tasks, single image desnowing is a more challenging task. This is mainly due to the diversity and irregularity of snow shape, which makes it extremely difficult to restore images in snowy scenes. Moreover, snow particles also have a veiling effect similar to haze or mist. Although current works can effectively remove snow particles with various shapes, they also bring distortion to the restored image. To address these issues, we propose a novel single image desnowing network called Star-Net. First, we design a Star type Skip Connection (SSC) to establish information channels for all different scale features, which can deal with the complex shape of snow particles.Second, we present a Multi-Stage Interactive Transformer (MIT) as the base module of Star-Net, which is designed to better understand snow particle shapes and to address image distortion by explicitly modeling a variety of important image recovery features. Finally, we propose a Degenerate Filter Module (DFM) to filter the snow particle and snow fog residual in the SSC on the spatial and channel domains. Extensive experiments show that our Star-Net achieves state-of-the-art snow removal performances on three standard snow removal datasets and retains the original sharpness of the images.
翻译:相较于其他严重气象影像修复任务,单幅图像去雪是一项更具挑战性的任务。这主要是因为雪的形状多样且不规则,导致在雪景中修复图像极其困难。此外,雪粒子还具有类似于雾霾或雾气的遮盖效应。尽管当前的工作可以有效地去除具有各种形状的雪粒子,但它们也会给恢复图像带来失真。为了解决这些问题,我们提出了一种新的单幅图像去雪网络称为“星型网络”(Star-Net)。首先,我们设计了一种星形跳跃连接(SSC)来为所有不同尺度的特征建立信息通道,这可以处理雪粒子的复杂形状。其次,我们提出了一个多阶段交互变压器(MIT)作为Star-Net的基本模块,旨在更好地理解雪粒子形状,并通过显式建模各种重要图像恢复特征来解决图像失真问题。最后,我们提出了一种退化滤波器模块(DFM),以在空间和通道域中过滤SSC中的雪粒子和雪雾残留物。广泛的实验证明,我们的Star-Net在三个标准去雪数据集上实现了最先进的去雪性能,并保留了原始图像的清晰度。