Removing background noise from speech audio has been the subject of considerable effort, especially in recent years due to the rise of virtual communication and amateur recordings. Yet background noise is not the only unpleasant disturbance that can prevent intelligibility: reverb, clipping, codec artifacts, problematic equalization, limited bandwidth, or inconsistent loudness are equally disturbing and ubiquitous. In this work, we propose to consider the task of speech enhancement as a holistic endeavor, and present a universal speech enhancement system that tackles 55 different distortions at the same time. Our approach consists of a generative model that employs score-based diffusion, together with a multi-resolution conditioning network that performs enhancement with mixture density networks. We show that this approach significantly outperforms the state of the art in a subjective test performed by expert listeners. We also show that it achieves competitive objective scores with just 4-8 diffusion steps, despite not considering any particular strategy for fast sampling. We hope that both our methodology and technical contributions encourage researchers and practitioners to adopt a universal approach to speech enhancement, possibly framing it as a generative task.
翻译:在这项工作中,我们提议将语音增强任务视为一项整体努力,并同时提出一个处理55种不同扭曲现象的普遍语音强化系统。我们的方法包括一种基于分数的传播模式,以及一个与混合密度网络进行增强的多分辨率调节网络。我们表明,这种方法在专家听众进行的主观测试中大大优于艺术状态。我们还表明,尽管没有考虑任何快速取样的具体战略,但仅以4-8个传播步骤实现了竞争性目标分数。我们希望我们的方法和技术贡献都能够鼓励研究人员和从业人员对语音增强采取普遍做法,并可能将其描述为一种配方任务。