In this paper we propose a wavelet-based video codec specifically designed for VR displays that enables real-time playback of high-resolution 360{\deg} videos. Our codec exploits the fact that only a fraction of the full 360{\deg} video frame is visible on the display at any time. To load and decode the video viewport-dependently in real-time, we make use of the wavelet transform for intra- as well as inter-frame coding. Thereby, the relevant content is directly streamed from the drive, without the need to hold the entire frames in the memory. With an average of 190 frames per second at 8192x8192-pixel full frame resolution, the evaluations demonstrate that our codec's decoding performance is up to 272% higher than that of the state-of-the-art video codecs H.265 and AV1 for typical VR displays. Finally, we demonstrate how our wavelet-based codec can also directly be used in conjunction with foveation for further performance increases.
翻译：在本文中,我们建议为 VR 显示器专门设计一个波盘式视频编码器,使高分辨率360xdeg} 视频能够实时重播。 我们的编码器利用了一个事实,即完整360xdeg} 视频框架的一小部分随时都可以在显示器上看到。 要实时装载和解码视频显示器, 我们使用波盘转换器进行内部和内部编码。 因此, 相关内容直接从驱动器中流出, 不需要保持整个内存框架。 由于平均每秒190个框架, 平均为 8992x8192- pixel 完整框架解析, 评估显示我们的编码器解码性能比常规 VR 显示器的高级视频编码H. 265 和 AV1 高出272%。 最后, 我们演示我们的波盘调调码器如何直接用于进一步提升性能。
相关内容
微信扫码咨询专知VIP会员