Transformer-based image denoising methods have achieved encouraging results in the past year. However, it must uses linear operations to model long-range dependencies, which greatly increases model inference time and consumes GPU storage space. Compared with convolutional neural network-based methods, current Transformer-based image denoising methods cannot achieve a balance between performance improvement and resource consumption. In this paper, we propose an Efficient Wavelet Transformer (EWT) for image denoising. Specifically, we use Discrete Wavelet Transform (DWT) and Inverse Wavelet Transform (IWT) for downsampling and upsampling, respectively. This method can fully preserve the image features while reducing the image resolution, thereby greatly reducing the device resource consumption of the Transformer model. Furthermore, we propose a novel Dual-stream Feature Extraction Block (DFEB) to extract image features at different levels, which can further reduce model inference time and GPU memory usage. Experiments show that our method speeds up the original Transformer by more than 80%, reduces GPU memory usage by more than 60%, and achieves excellent denoising results. All code will be public.
翻译:基于Transformer的图像去噪方法在过去一年中取得了令人鼓舞的结果。然而,它必须使用线性操作来模拟长距离的依赖关系,这大大增加了模型推理时间和GPU存储空间的消耗。与卷积神经网络相比,当前的基于Transformer的图像去噪方法无法在性能提高和资源消耗之间达到平衡。在本文中,我们提出了一种高效的小波Transformer(EWT)用于图像去噪。具体而言,我们使用离散小波变换(DWT)和逆小波变换(IWT)进行下采样和上采样。这种方法可以充分保留图像特征,同时降低图像分辨率,从而大大减少Transformer模型的计算资源消耗。此外,我们提出了一种新颖的双流特征提取模块(DFEB)来提取不同层次的图像特征,进一步减少模型推理时间和GPU内存使用量。实验结果表明,我们的方法将原始Transformer的速度提高了80%以上,将GPU内存使用量降低了60%以上,并取得了出色的去噪效果。所有代码将公开。