Diffusion models are a class of generative models that learn to synthesize samples by inverting a diffusion process that gradually maps data into noise. While these models have enjoyed great success recently, a full theoretical understanding of their observed properties is still lacking, in particular, their weak sensitivity to the choice of noise family and the role of adequate scheduling of noise levels for good synthesis. By identifying a correspondence between diffusion models and a well-known paradigm in cognitive science known as serial reproduction, whereby human agents iteratively observe and reproduce stimuli from memory, we show how the aforementioned properties of diffusion models can be explained as a natural consequence of this correspondence. We then complement our theoretical analysis with simulations that exhibit these key features. Our work highlights how classic paradigms in cognitive science can shed light on state-of-the-art machine learning problems.
翻译:传播模型是一组基因化模型,通过将逐渐将数据映射成噪音的传播过程转换为扩散过程,学会合成样品。这些模型最近取得了巨大成功,但对于所观察到的特性仍然缺乏全面的理论理解,特别是它们对噪音家庭的选择以及适当安排噪音水平以便进行良好合成的作用缺乏充分的敏感性。通过确定传播模型与认知科学中众所周知的称为序列复制的范例之间的对应关系,通过这种模式,人类代理机可以迭代地观察并复制记忆中的刺激性,我们展示了上述扩散模型的特性如何被解释为这些通信的自然后果。然后我们用展示这些关键特征的模拟来补充我们的理论分析。我们的工作突出强调了认知科学的典型模式如何能够揭示最先进的机器学习问题。