Progressively applying Gaussian noise transforms complex data distributions to approximately Gaussian. Reversing this dynamic defines a generative model. When the forward noising process is given by a Stochastic Differential Equation (SDE), Song et al. (2021) demonstrate how the time inhomogeneous drift of the associated reverse-time SDE may be estimated using score-matching. A limitation of this approach is that the forward-time SDE must be run for a sufficiently long time for the final distribution to be approximately Gaussian. In contrast, solving the Schr\"odinger Bridge problem (SB), i.e. an entropy-regularized optimal transport problem on path spaces, yields diffusions which generate samples from the data distribution in finite time. We present Diffusion SB (DSB), an original approximation of the Iterative Proportional Fitting (IPF) procedure to solve the SB problem, and provide theoretical analysis along with generative modeling experiments. The first DSB iteration recovers the methodology proposed by Song et al. (2021), with the flexibility of using shorter time intervals, as subsequent DSB iterations reduce the discrepancy between the final-time marginal of the forward (resp. backward) SDE with respect to the prior (resp. data) distribution. Beyond generative modeling, DSB offers a widely applicable computational optimal transport tool as the continuous state-space analogue of the popular Sinkhorn algorithm (Cuturi, 2013).
翻译:扩散薛定谔桥应用于基于分数的生成建模
翻译后的摘要:
通过逐渐应用高斯噪声将复杂数据分布转换为近似高斯分布,倒转这个动态定义一个生成模型。当正向加噪过程由随机微分方程(SDE)给出时,Song等人(2021)演示了如何使用评分匹配来估计相关的逆时间SDE的时间不均匀漂移。这种方法的局限性在于,正向时间SDE必须运行足够长的时间才能使最终分布近似于高斯分布。相反,解决Schr \"odinger桥难题(SB),即在路径空间上的熵正则化最优传输问题,产生的扩散可在有限时间内从数据分布中生成样本。我们提出Diffusion SB(DSB),一种原始的近似迭代比例拟合(IPF)过程来解决SB问题,并提供理论分析以及生成建模实验。第一个DSB迭代恢复了Song等人(2021)提出的方法,随着随后的DSB迭代,前向(反向)SDE的最终时间边际相对于先验(或数据)分布之间的差异减小。除了生成建模,DSB还提供了一种广泛适用的计算最优传输工具,作为流行的Sinkhorn算法(Cuturi,2013)的连续状态空间模拟。