Granular sound synthesis is a popular audio generation technique based on rearranging sequences of small waveform windows. In order to control the synthesis, all grains in a given corpus are analyzed through a set of acoustic descriptors. This provides a representation reflecting some form of local similarities across the grains. However, the quality of this grain space is bound by that of the descriptors. Its traversal is not continuously invertible to signal and does not render any structured temporality. We demonstrate that generative neural networks can implement granular synthesis while alleviating most of its shortcomings. We efficiently replace its audio descriptor basis by a probabilistic latent space learned with a Variational Auto-Encoder. In this setting the learned grain space is invertible, meaning that we can continuously synthesize sound when traversing its dimensions. It also implies that original grains are not stored for synthesis. Another major advantage of our approach is to learn structured paths inside this latent space by training a higher-level temporal embedding over arranged grain sequences. The model can be applied to many types of libraries, including pitched notes or unpitched drums and environmental noises. We report experiments on the common granular synthesis processes as well as novel ones such as conditional sampling and morphing.
翻译:粒子合成是一种以小波形窗口的重新排列序列为基础的流行音频生成技术。 为了控制合成, 通过一组声学描述器对特定元素中的所有粒子进行分析。 这代表了各个粒子之间的某种地方相似性。 但是, 这个粒子空间的质量是受描述器的束缚的。 它的穿孔不会不断地被忽略, 也不会造成任何结构化的时间性。 我们证明基因神经网络可以实施颗粒合成, 同时减轻其大部分缺点。 为了控制合成, 我们通过一套动态自动电解器来有效地用一个概率隐蔽空间来取代其音频描述器基础。 在此设置中, 所学的谷物空间是不可逆的, 意思是, 在穿孔时我们可以持续合成它的声音。 它还意味着原始的粒子不会被存储来进行合成。 我们的方法的另一个主要优点是, 通过训练高层次的时间嵌入于所安排的粒子序列, 来学习这个隐蔽的路径。 这个模型可以应用于许多种类的图书馆, 包括嵌入式的笔记或无刺的圆形隐蔽的隐蔽的隐蔽的隐形器和环境变压器。 我们作为普通的合成的合成的磁制的合成过程。