Volumetric scene representations enable photorealistic view synthesis for static scenes and form the basis of several existing 6-DoF video techniques. However, the volume rendering procedures that drive these representations necessitate careful trade-offs in terms of quality, rendering speed, and memory efficiency. In particular, existing methods fail to simultaneously achieve real-time performance, small memory footprint, and high-quality rendering for challenging real-world scenes. To address these issues, we present HyperReel -- a novel 6-DoF video representation. The two core components of HyperReel are: (1) a ray-conditioned sample prediction network that enables high-fidelity, high frame rate rendering at high resolutions and (2) a compact and memory efficient dynamic volume representation. Our 6-DoF video pipeline achieves the best performance compared to prior and contemporary approaches in terms of visual quality with small memory requirements, while also rendering at up to 18 frames-per-second at megapixel resolution without any custom CUDA code.
翻译:体积场景展示能够对静态场景进行摄影现实观合成,并形成若干现有的6-DoF视频技术的基础。然而,驱动这些场景的量化程序要求这些场景在质量、速度和记忆效率方面进行谨慎的权衡,特别是,现有方法未能同时实现实时性能、小记忆足迹和高质量的真实世界场景挑战性。为解决这些问题,我们展示了超视距 -- -- 6-DoF视频新版本。超视距的两个核心部分是:(1)一个光质化样本预测网络,能够实现高纤维性、高框架率高分辨率,(2)一个紧凑和记忆动态的体积代表。我们的6-DoF视频管道在视觉质量方面实现了与以往和当代方法相比的最佳性能,同时满足了小记忆要求,同时在没有定制 CUDA 代码的情况下,在巨像分辨率上将每秒18个框架设定为每秒。