This work studies discrete diffusion probabilistic models with applications to natural language generation. We derive an alternative yet equivalent formulation of the sampling from discrete diffusion processes and leverage this insight to develop a family of reparameterized discrete diffusion models. The derived generic framework is highly flexible, offers a fresh perspective of the generation process in discrete diffusion models, and features more effective training and decoding techniques. We conduct extensive experiments to evaluate the text generation capability of our model, demonstrating significant improvements over existing diffusion models.
翻译:这项工作研究离散扩散概率模型和自然语言生成的应用。我们从离散扩散过程中获得替代的、但等效的样本配方,并利用这一洞察力发展一套再生的离散扩散模型。衍生的通用框架非常灵活,在离散扩散模型中提供了生成过程的新视角,并具有更有效的培训和解码技术。我们进行了广泛的实验,评估了我们模型的文本生成能力,展示了与现有扩散模型相比的重大改进。