Video super-resolution has recently become one of the most important mobile-related problems due to the rise of video communication and streaming services. While many solutions have been proposed for this task, the majority of them are too computationally expensive to run on portable devices with limited hardware resources. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based video super-resolution solutions that can achieve a real-time performance on mobile GPUs. The participants were provided with the REDS dataset and trained their models to do an efficient 4X video upscaling. The runtime of all models was evaluated on the OPPO Find X2 smartphone with the Snapdragon 865 SoC capable of accelerating floating-point networks on its Adreno GPU. The proposed solutions are fully compatible with any mobile GPU and can upscale videos to HD resolution at up to 80 FPS while demonstrating high fidelity results. A detailed description of all models developed in the challenge is provided in this paper.
翻译:由于视频通信和流流服务的兴起,视频超级分辨率最近已成为最重要的移动相关问题之一。虽然为这项任务提出了许多解决方案,但大多数解决方案的计算成本太高,无法在硬件资源有限的便携式设备上运行。为解决这一问题,我们引入了第一个移动AI挑战,目标是开发一个端至端深层次的基于学习的视频超级分辨率解决方案,在移动GPU上实现实时性能。向参与者提供了REDS数据集,并培训了他们的模型,以便高效地进行4X视频升级。所有模型的运行时间都是在OPPO Find X2智能手机上与能够加速其Adreno GPU上浮动点网络的 Sentragy 865 SoC 上进行的评估。拟议解决方案与任何移动GPU完全兼容,可以在80个FPS上将视频升级到HD分辨率,同时展示高度忠诚的结果。本文详细说明了在挑战中开发的所有模型。