We present Imagen Video, a text-conditional video generation system based on a cascade of video diffusion models. Given a text prompt, Imagen Video generates high definition videos using a base video generation model and a sequence of interleaved spatial and temporal video super-resolution models. We describe how we scale up the system as a high definition text-to-video model including design decisions such as the choice of fully-convolutional temporal and spatial super-resolution models at certain resolutions, and the choice of the v-parameterization of diffusion models. In addition, we confirm and transfer findings from previous work on diffusion-based image generation to the video generation setting. Finally, we apply progressive distillation to our video models with classifier-free guidance for fast, high quality sampling. We find Imagen Video not only capable of generating videos of high fidelity, but also having a high degree of controllability and world knowledge, including the ability to generate diverse videos and text animations in various artistic styles and with 3D object understanding. See https://imagen.research.google/video/ for samples.
翻译:我们展示了图像视频,这是一个基于一系列视频传播模型的文本条件视频生成系统。根据一个文本提示,图像视频使用一个基础视频生成模型和一系列空间和时间间视频超分辨率模型生成高定义视频。我们描述我们如何扩大该系统,将其作为高定义文本到视频模型,包括在某些分辨率上选择完全进化时间和空间超分辨率模型的设计决定,以及选择扩散模型的副参数。此外,我们还确认和将以往关于基于传播的图像生成工作的调查结果传输到视频生成设置。最后,我们用快速、高质量的分类式无指导对视频模型进行渐进蒸馏。我们发现图像视频不仅能够生成高度忠诚的视频,而且具有高度控制性和世界知识,包括以各种艺术风格和3D对象理解生成多种视频和文本动画的能力。见 https://imagen.research.google/vicion/ for sample。