Fast accuracy estimation of deep learning based multi-class musical source separation

Music source separation represents the task of extracting all the instruments from a given song. Recent breakthroughs on this challenge have gravitated around a single dataset, MUSDB, limited to four instrument classes only. Larger datasets and more instruments are costly and time-consuming in collecting data and training deep neural networks (DNNs). In this work, we propose a fast method to evaluate the separability of instruments in any dataset without training and tuning a DNN. This separability measure helps to select appropriate samples for the efficient training of neural networks. Based on the oracle principle with an ideal ratio mask, our approach is an excellent proxy to estimate the separation performances of state-of-the-art deep learning approaches based on Time-Frequency (TF) masking such as TasNet or Open-Unmix. Our results indicate that even if TasNet has the freedom to learn a latent space where instruments would be separated efficiently by a masking process, this space is no better than the TF representation.