GALD-SE: Guided Anisotropic Lightweight Diffusion for Efficient Speech Enhancement

Speech enhancement is designed to enhance the intelligibility and quality of speech across diverse noise conditions. Recently, diffusion model has gained lots of attention in speech enhancement area, achieving competitive results. Current diffusion-based methods blur the initial signal with isotropic Gaussian noise and recover clean speech from the prior. However, these methods often suffer from a substantial computational burden. We argue that the inefficiency stems from the oversight that speech enhancement is not purely a generative task; it primarily involves noise reduction and completion of missing information, while the clean clues in the original mixture do not need to be regenerated. In this paper, we propose a method that introduces noise with anisotropic guidance during the diffusion process, allowing the neural network to focus on clean clues within noisy recordings. This approach is robust against various types of noise interference and speech distortion, and significantly reduces the computational load. Experiments demonstrate that the proposed method achieves state-of-the-art results with only approximately 4.5 million parameters, significantly fewer than those required by other diffusion methods. This effectively narrows the gap in model size between diffusion-based and predictive speech enhancement approaches. Additionally, the proposed method performs well in very noisy scenarios, demonstrating its potential for applications in highly challenging environments.