Free-viewpoint video (FVV) enables immersive viewing experiences by allowing users to view scenes from arbitrary perspectives. As a prominent reconstruction technique for FVV generation, 4D Gaussian Splatting (4DGS) models dynamic scenes with time-varying 3D Gaussian ellipsoids and achieves high-quality rendering via fast rasterization. However, existing 4DGS approaches suffer from quality degradation over long sequences and impose substantial bandwidth and storage overhead, limiting their applicability in real-time and wide-scale deployments. Therefore, we present AirGS, a streaming-optimized 4DGS framework that rearchitects the training and delivery pipeline to enable high-quality, low-latency FVV experiences. AirGS converts Gaussian video streams into multi-channel 2D formats and intelligently identifies keyframes to enhance frame reconstruction quality. It further combines temporal coherence with inflation loss to reduce training time and representation size. To support communication-efficient transmission, AirGS models 4DGS delivery as an integer linear programming problem and design a lightweight pruning level selection algorithm to adaptively prune the Gaussian updates to be transmitted, balancing reconstruction quality and bandwidth consumption. Extensive experiments demonstrate that AirGS reduces quality deviation in PSNR by more than 20% when scene changes, maintains frame-level PSNR consistently above 30, accelerates training by 6 times, reduces per-frame transmission size by nearly 50% compared to the SOTA 4DGS approaches.
翻译：自由视点视频（FVV）允许用户从任意视角观看场景，从而实现沉浸式观看体验。作为FVV生成的重要重建技术，4D高斯泼溅（4DGS）通过时变的三维高斯椭球体对动态场景进行建模，并借助快速光栅化实现高质量渲染。然而，现有4DGS方法在长序列上存在质量退化问题，且带来显著的带宽与存储开销，限制了其在实时与大规模部署中的应用。为此，我们提出AirGS——一个面向流式传输优化的4DGS框架，通过重构训练与传输管线以实现高质量、低延迟的FVV体验。AirGS将高斯视频流转换为多通道二维格式，并智能识别关键帧以提升帧重建质量。该方法进一步结合时序连贯性与膨胀损失，以降低训练时间与表示规模。为支持通信高效的传输，AirGS将4DGS传输建模为整数线性规划问题，并设计轻量级剪枝层级选择算法，自适应地剪裁待传输的高斯更新量，从而平衡重建质量与带宽消耗。大量实验表明：在场景变化时，AirGS将PSNR质量偏差降低超过20%；帧级PSNR持续稳定在30以上；与最先进的4DGS方法相比，训练速度提升6倍，单帧传输数据量减少近50%。
相关内容
微信扫码咨询专知VIP会员