This paper explores an efficient solution for Space-time Super-Resolution, aiming to generate High-resolution Slow-motion videos from Low Resolution and Low Frame rate videos. A simplistic solution is the sequential running of Video Super Resolution and Video Frame interpolation models. However, this type of solutions are memory inefficient, have high inference time, and could not make the proper use of space-time relation property. To this extent, we first interpolate in LR space using quadratic modeling. Input LR frames are super-resolved using a state-of-the-art Video Super-Resolution method. Flowmaps and blending mask which are used to synthesize LR interpolated frame is reused in HR space using bilinear upsampling. This leads to a coarse estimate of HR intermediate frame which often contains artifacts along motion boundaries. We use a refinement network to improve the quality of HR intermediate frame via residual learning. Our model is lightweight and performs better than current state-of-the-art models in REDS STSR Validation set.
翻译:本文探讨了空间时超分辨率的高效解决方案,目的是从低分辨率和低框架速率视频中生成高分辨率慢动视频。一个简单化解决方案是连续运行视频超分辨率和视频框架内插模型。然而,这类解决方案的记忆效率低,推断时间长,无法适当使用空间-时间关系属性。在这方面,我们首先使用二次模型在远程空间进行干涉。输入的远程框架是使用最先进的视频超分辨率和低框架速分辨率视频生成的高分辨率慢动视频。用于合成远程超分辨率和视频框架内插框架的流程图和混合面罩,在人力资源空间使用双线上加插图和混合面罩进行再利用。这导致对人力资源中间框架的粗略估计,其中往往含有运动边界上的文物。我们使用精细的网络通过残余学习提高人力资源中间框架的质量。我们的模型比REDS STSR 校验的当前最新模型轻度和表现得更好。