We present a new framework SoundDet, which is an end-to-end trainable and light-weight framework, for polyphonic moving sound event detection and localization. Prior methods typically approach this problem by preprocessing raw waveform into time-frequency representations, which is more amenable to process with well-established image processing pipelines. Prior methods also detect in segment-wise manner, leading to incomplete and partial detections. SoundDet takes a novel approach and directly consumes the raw, multichannel waveform and treats the spatio-temporal sound event as a complete "sound-object" to be detected. Specifically, SoundDet consists of a backbone neural network and two parallel heads for temporal detection and spatial localization, respectively. Given the large sampling rate of raw waveform, the backbone network first learns a set of phase-sensitive and frequency-selective bank of filters to explicitly retain direction-of-arrival information, whilst being highly computationally and parametrically efficient than standard 1D/2D convolution. A dense sound event proposal map is then constructed to handle the challenges of predicting events with large varying temporal duration. Accompanying the dense proposal map are a temporal overlapness map and a motion smoothness map that measure a proposal's confidence to be an event from temporal detection accuracy and movement consistency perspective. Involving the two maps guarantees SoundDet to be trained in a spatio-temporally unified manner. Experimental results on the public DCASE dataset show the advantage of SoundDet on both segment-based and our newly proposed event-based evaluation system.
翻译:我们提出了一个新框架“SoundDet ”, 这是一种端到端的、可训练的和轻量级的框架, 用于多声移动事件探测和本地化。 先前的方法通常通过将原始波形预处理成时间-频率显示器来处理这一问题, 更便于用固定的图像处理管道进行处理。 先前的方法也以片段方式探测, 导致不完全和部分检测。 声音Dot 采取新颖的方法, 直接使用原始的、 多通道波形, 并直接使用原生的、 多通道波形, 将片段- 时空声音事件作为完整的“ 声音- 目标” 处理, 有待检测。 具体地说, SoundDet 包括一个坚固的神经网络, 以及两个平行的线状显示时间- 时间- 测试和空间定位的平行头, 鉴于原始波段的取样率很大, 骨干网络首先学习一套对阶段性、 选择性的过滤库, 明确保留抵达方向信息, 同时高度的计算和对准效率高于标准 1D/2D 。 然后构建一个密性的建议地图上, 正在构建一个精确的精确的地图, 测量图上的一个测量度显示一个对时间测量度的精确度 。 。