We present the first neural video compression method based on generative adversarial networks (GANs). Our approach significantly outperforms previous neural and non-neural video compression methods in a user study, setting a new state-of-the-art in visual quality for neural methods. We show that the GAN loss is crucial to obtain this high visual quality. Two components make the GAN loss effective: we i) synthesize detail by conditioning the generator on a latent extracted from the warped previous reconstruction to then ii) propagate this detail with high-quality flow. We find that user studies are required to compare methods, i.e., none of our quantitative metrics were able to predict all studies. We present the network design choices in detail, and ablate them with user studies.
翻译:我们以基因对抗网络(GANs)为基础提出第一种神经视频压缩方法。我们的方法在用户研究中大大优于先前的神经和非神经视频压缩方法,为神经方法的视觉质量设定了新的最新水平。我们表明,GAN损失对于获得这种高视觉质量至关重要。有两个组成部分使得GAN损失有效:一)通过对从先前扭曲的重建中提取的潜伏的电源进行调节来综合细节,到那时为止;二)以高质量的流量传播这一细节。我们发现,用户研究需要比较方法,即我们没有定量指标能够预测所有研究。我们详细介绍了网络设计选择,并将它们与用户研究联系起来。