As an important application form of immersive multimedia services, free-viewpoint video(FVV) enables users with great immersive experience by strong interaction. However, the computational complexity of virtual view synthesis algorithms poses a significant challenge to the real-time performance of an FVV system. Furthermore, the individuality of user interaction makes it difficult to serve multiple users simultaneously for a system with conventional architecture. In this paper, we novelly introduce a CNN-based view interpolation algorithm to synthesis dense virtual views in real time. Based on this, we also build an end-to-end live free-viewpoint system with a multi-user oriented streaming strategy. Our system can utilize a single edge server to serve multiple users at the same time without having to bring a large view synthesis load on the client side. We analysis the whole system and show that our approaches give the user a pleasant immersive experience, in terms of both visual quality and latency.
翻译：作为隐性多媒体服务的重要应用形式,自由视点视频(FVV)使通过强力互动具有丰富亲身体验的用户得以使用。然而,虚拟视图合成算法的计算复杂性对虚拟视图合成算法的实时性能提出了重大挑战。此外,用户互动的个性使得难以同时为多个用户提供具有常规架构的系统服务。在本文中,我们新引入了基于CNN的视觉内插算法,实时合成密集的虚拟视图。在此基础上,我们还建立了一个以多用户为导向的流动战略的端到端现场自由点系统。我们的系统可以使用单一的边缘服务器为多个用户同时服务,而不必在客户方面带来一个大视角的合成负载。我们分析了整个系统,并展示了我们的做法给用户在视觉质量和长期性两方面都带来了愉快的亲切体验。
相关内容
微信扫码咨询专知VIP会员