Diffusion models in the literature are optimized with various objectives that are special cases of a weighted loss, where the weighting function specifies the weight per noise level. Uniform weighting corresponds to maximizing the ELBO, a principled approximation of maximum likelihood. In current practice diffusion models are optimized with non-uniform weighting due to better results in terms of sample quality. In this work we expose a direct relationship between the weighted loss (with any weighting) and the ELBO objective. We show that the weighted loss can be written as a weighted integral of ELBOs, with one ELBO per noise level. If the weighting function is monotonic, then the weighted loss is a likelihood-based objective: it maximizes the ELBO under simple data augmentation, namely Gaussian noise perturbation. Our main contribution is a deeper theoretical understanding of the diffusion objective, but we also performed some experiments comparing monotonic with non-monotonic weightings, finding that monotonic weighting performs competitively with the best published results.
翻译:文献中的传播模型以各种特殊的目标优化,这些目标是加权损失的特例,加权函数指定每噪声水平的重量。统一加权对应最大程度的ELBO,这是最大可能性的一个原则近似值。在目前的做法中,扩散模型以非统一加权法优化,因为抽样质量方面的结果更好。在这项工作中,我们暴露了加权损失(与任何加权)与ELBO目标之间的直接关系。我们显示,加权损失可以写成ELBOs的加权组成部分,每噪声水平有一个ELBO。如果加权功能是单调的,那么加权损失则是一个基于可能性的目标:在简单数据增强的情况下,即高斯噪音渗透,使ELBO最大化。我们的主要贡献是对扩散目标的更深的理论理解,但我们也进行了一些实验,将单调与非调重力比较,发现单调权重与最佳公布的结果具有竞争力。</s>