Lune

CVPR2024Top-tier venue

4D Gaussian Splatting for Real-Time Dynamic Scene Rendering

Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, Xinggang Wang

Abstract

Representing and rendering dynamic scenes has been an important but challenging task. Especially, to accurately model complex motions, high efficiency is usually hard to guarantee. To achieve real-time dynamic scene rendering while also enjoying high training and storage efficiency, we propose 4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes rather than applying 3D-GS for each individual frame. In 4D-GS, a novel explicit representation containing both 3D Gaussians and 4D neural voxels is proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is proposed to efficiently build Gaussian features from 4D neural voxels and then a lightweight MLP is applied to predict Gaussian deformations at novel timestamps. Our 4D-GS method achieves real-time rendering under high resolutions, 82 FPS at an 800x800 resolution on an RTX 3090 GPU while maintaining comparable or better quality than previous state- of-the-art methods. More demos and code are available at https://guanjunwu.github.io/4dgs/.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

336 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding cites Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
  • OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding cites Instant neural graphics primitives with a multiresolution hash encoding
  • OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
  • OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding cites Plenoxels: Radiance Fields without Neural Networks
  • GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
  • Gaussian Splatting SLAM cites Instant neural graphics primitives with a multiresolution hash encoding
  • Gaussian Splatting SLAM cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
  • Gaussian Splatting SLAM cites Plenoxels: Radiance Fields without Neural Networks
  • Instant neural graphics primitives with a multiresolution hash encoding cites Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
  • Segment Anything in 3D with NeRFs cites Instant neural graphics primitives with a multiresolution hash encoding
  • Segment Anything in 3D with NeRFs cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
  • Segment Anything in 3D with NeRFs cites Plenoxels: Radiance Fields without Neural Networks
  • 3D Gaussian Splatting for Real-Time Radiance Field Rendering cites Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
  • 3D Gaussian Splatting for Real-Time Radiance Field Rendering cites Instant neural graphics primitives with a multiresolution hash encoding
  • 3D Gaussian Splatting for Real-Time Radiance Field Rendering cites Plenoxels: Radiance Fields without Neural Networks
  • 4Diffusion: Multi-view Video Diffusion Model for 4D Generation cites Instant neural graphics primitives with a multiresolution hash encoding
  • 4Diffusion: Multi-view Video Diffusion Model for 4D Generation cites 3D Gaussian Splatting for Real-Time Radiance Field Rendering
  • Plenoxels: Radiance Fields without Neural Networks cites Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers336

  1. Gaussian Splatting SLAMHidenobu Matsuki, Riku Murai, Paul H. J. Kelly, Andrew J. DavisonCVPR 2024 · 328 citations
  2. GS-SLAM: Dense Visual SLAM with 3D Gaussian SplattingChi Yan, Delin Qu, Dan Xu, Bin Zhao et al.CVPR 2024 · 270 citations
  3. Segment Anything in 3D with NeRFsJiazhong Cen, Zanwei Zhou, Jiemin Fang, Chen Yang et al.NeurIPS 2023 · 255 citations
  4. OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary UnderstandingYanmin Wu, Jiarui Meng, Haijie Li, Chenming Wu et al.NeurIPS 2024 · 191 citations
  5. 4Diffusion: Multi-view Video Diffusion Model for 4D GenerationHaiyu Zhang, Xinyuan Chen, Yaohui Wang, Xihui Liu et al.NeurIPS 2024 · 119 citations

Builds on28

  1. 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
  2. Instant neural graphics primitives with a multiresolution hash encodingThomas Müller, Alex Evans, Christoph Schied, Alexander KellerSIGGRAPH 2022 · 4,089 citations
  3. Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance FieldsJonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman et al.ICCV 2021 · 2,700 citations
  4. Nerfies: Deformable Neural Radiance FieldsKeunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz et al.ICCV 2021 · 1,442 citations
  5. Plenoxels: Radiance Fields without Neural NetworksSara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen et al.CVPR 2022 · 1,237 citations

Related papers

  1. ST-4DGS: Spatial-Temporally Consistent 4D Gaussian Splatting for Efficient Dynamic Scene RenderingDeqi Li, Shi-Sheng Huang, Zhiyuan Lu, Xinran Duan et al.SIGGRAPH 2024 · 33 citations
  2. Grid4D: 4D Decomposed Hash Encoding for High-Fidelity Dynamic Gaussian SplattingJiawei Xu, Zexin Fan, Jian Yang, Jin XieNeurIPS 2024 · 64 citations
  3. Fully Explicit Dynamic Gaussian SplattingJunoh Lee, Changyeon Won, Hyunjun Jung, Inhwan Bae et al.NeurIPS 2024 · 92 citations
  4. DASH: 4D Hash Encoding with Self-Supervised Decomposition for Real-Time Dynamic Scene RenderingJie Chen, Zhangchi Hu, Peixi Wu, Huyue Zhu et al.ICCV 2025 · 2 citations
  5. DN-4DGS: Denoised Deformable Network with Temporal-Spatial Aggregation for Dynamic Scene RenderingJiahao Lu, Jiacheng Deng, Ruijie Zhu, Yanzhe Liang et al.NeurIPS 2024 · 37 citations
Dusk over the sea between two cliffs drawn in fine vertical lines