Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables to learn and discover latent relationship between interesting lyrics and accompanying melody. Unfortunately, the limited availability of paired lyrics-melody dataset with alignment information has hindered the research progress. To address this problem, we create a large dataset consisting of 12,197 MIDI songs each with paired lyrics and melody alignment through leveraging different music sources where alignment relationship between syllables and music attributes is extracted. Most importantly, we propose a novel deep generative model, conditional Long Short-Term Memory - Generative Adversarial Network (LSTM-GAN) for melody generation from lyrics, which contains a deep LSTM generator and a deep LSTM discriminator both conditioned on lyrics. In particular, lyrics-conditioned melody and alignment relationship between syllables of given lyrics and notes of predicted melody are generated simultaneously. Experimental results have proved the effectiveness of our proposed lyrics-to-melody generative model, where plausible and tuneful sequences can be inferred from lyrics.
翻译:在人工智能和音乐领域,从歌词中产生梅乐蒂的一代是一个具有挑战性的研究问题,它使得能够学习和发现有趣的歌词和伴奏旋律之间的潜在关系。不幸的是,配对歌词和调和信息的可用性有限阻碍了研究进展。为了解决这一问题,我们通过利用不同音乐来源,在调和音调关系中提取调和音调调调,创建了由12 197个配对歌词和调和组成的大型数据集。最重要的是,我们为歌词的旋律一代提出了一个新型的深层次基因模型,即有条件的短期内存 -- -- 吉他式Adversarial 网络(LSTM-GAN),它包含一个深 LSTM 生成器和以歌词为条件的深 LSTM 区分器。特别是歌词和预言调调调调调调调调和调和调和调和调调调,同时产生。实验结果证明了我们提议的歌词调调调调调和调调调调调调的组合模式的有效性,从歌词调中可以推断出合理和调和调和的顺序。