EdgeGaussian: Real-time Free-Viewpoint Video for Mobile VR via Edge-Client Collaborative Neural Rendering

Free-Viewpoint Videos (FVVs) enable immersive viewing of a scene from any position and angle using virtual reality (VR) head-mounted displays (HMDs), thus have great potential in various applications such as telepresence, gaming, and education. Recently, 3D Gaussian Splatting (3DGS) has emerged as a promising method for FVV construction due to its superior reconstruction quality. However, its real-time rendering on untethered HMDs remains challenging due to high computational demands. To address this challenge, we propose EdgeGaussian, a novel edge-client collaborative framework for real-time FVV rendering. Our approach employs decomposed static-dynamic 4D Gaussian splatting (SD-4DGS) to separately reconstruct static and dynamic components of a scene. We further introduce a hybrid mesh-4DGS neural representation, where static components are modeled as textured meshes for local rendering, while dynamic components are offloaded to edge servers as 4DGS. This decomposition significantly reduces the computational burden on the client device while maintaining high rendering quality. Our testbed experiments demonstrate that Edge-Gaussian achieves up to 128 FPS, outperforming state-of-the-art local rendering methods by 4x and edge rendering methods by 5x.