Score-based generative models (SGMs) have recently emerged as a promising class of generative models. However, a fundamental limitation is that their inference is very slow due to a need for many (e.g., 2000) iterations of sequential computations. An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation. We investigate this problem by viewing the diffusion sampling process as a Metropolis adjusted Langevin algorithm, which helps reveal the underlying cause to be ill-conditioned curvature. Under this insight, we propose a model-agnostic preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem. Crucially, PDS is proven theoretically to converge to the original target distribution of a SGM, no need for retraining. Extensive experiments on three image datasets with a variety of resolutions and diversity validate that PDS consistently accelerates off-the-shelf SGMs whilst maintaining the synthesis quality. In particular, PDS can accelerate by up to 29x on more challenging high resolution (1024x1024) image generation.
翻译:最近,基于分数的基因变异模型(SGMs)作为一种很有希望的基因变异模型(SGMs)出现了,但是,一个基本的限制是,由于需要许多序列计算(例如2000年)的迭代,它们的推论非常缓慢。一种直观的加速方法是减少抽样迭代,而这种迭代却导致严重性能退化。我们通过将扩散采样过程看成大都会调整的朗埃文算法来调查这一问题,这种算法有助于揭示造成曲调不良的根本原因。根据这种观察,我们建议采用一种模型化的、有先决条件的传播采样(PDS)方法,利用矩阵作为减轻上述问题的前提条件。奇怪的是,PDSS在理论上证明从理论上可以与SGM原目标分布汇合而无需再培训。对三个图像数据集进行了广泛的实验,其分辨率和多样性都证实PDSS在保持合成质量的同时,不断加速现成的SGMs。特别是PDS可以加速到29x,在更具挑战性的高分辨率(1024x1024)的图像生成上达到29。