This paper provide several mathematical analyses of the diffusion model in machine learning. The drift term of the backwards sampling process is represented as a conditional expectation involving the data distribution and the forward diffusion. The training process aims to find such a drift function by minimizing the mean-squared residue related to the conditional expectation. Using small-time approximations of the Green's function of the forward diffusion, we show that the analytical mean drift function in DDPM and the score function in SGM asymptotically blow up in the final stages of the sampling process for singular data distributions such as those concentrated on lower-dimensional manifolds, and is therefore difficult to approximate by a network. To overcome this difficulty, we derive a new target function and associated loss, which remains bounded even for singular data distributions. We illustrate the theoretical findings with several numerical examples.
翻译:本文对机器学习中的传播模型提供了数种数学分析。 向后取样过程的漂移术语是数据分布和前向扩散的一个有条件的预期。 培训过程的目的是通过尽量减少与有条件的预期有关的平均成份残留物来找到这种漂移功能。 使用绿色前向扩散功能的小型近似值,我们表明,DDPM的分析平均值漂移功能和SGM的分数函数在取样过程的最后阶段即时地爆炸,用于单一数据分布,例如那些集中在低维元上的数据分布,因此难以被网络所近似。 为了克服这一困难,我们产生了一个新的目标函数和相关损失,即使用于单一的数据分布,也仍然被捆绑在一起。 我们用几个数字例子来说明理论结论。