Decompositional Neural Scene Reconstruction with Generative Diffusion Prior
Junfeng Ni, Yu Liu, Ruijie Lu, Zirui Zhou, Song-Chun Zhu, Yixin Chen, Siyuan Huang
Abstract
Ours with 10 views ObjectSDF++ with 10 views ObjectSDF++ with 100 views Number of Views Object geometry editing Object appearance editing VFX editing Scene stylization Reconstructed mesh in Blender "Chinese style" "A round table" "Freeze it!" "Pokemon style" Figure 1 . We propose DP-RECON, which capitalizes on pre-trained diffusion models for complete and decompositional neural scene reconstruction. This approach significantly improves reconstruction quality in less captured regions, where previous methods often struggle. Additionally, our method enables flexible text-based editing of geometry and appearance, as well as photorealistic VFX editing.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers14
- ShapeR: Robust Conditional 3D Shape Generation from Casual CapturesYawar Siddiqui, Duncan P. Frost, Samir Aroudj, Armen Avetisyan et al.CVPR 2026 · 19 citations
- HoloScene: Simulation-Ready Interactive 3D Worlds from a Single VideoHongchi Xia, Chih-Hao Lin, Hao-Yu Hsu, Quentin Leboutet et al.NeurIPS 2025 · 18 citations
- Dexterous World ModelsByungjun Kim, Taeksoo Kim, Junyoung Lee, Hanbyul JooCVPR 2026 · 17 citations
- SimRecon: SimReady Compositional Scene Reconstruction from Real VideosChong Xia, Kai Zhu, Zizhuo Wang, Fangfu Liu et al.CVPR 2026 · 11 citations
- G4Splat: Geometry-Guided Gaussian Splatting with Generative PriorJunfeng Ni, Yixin Chen, Zhifei Yang, Yu Liu et al.ICLR 2026 · 10 citations
Builds on58
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Photorealistic Text-to-Image Diffusion Models with Deep Language UnderstandingChitwan Saharia, William Chan, Saurabh Saxena, Lala Li et al.NeurIPS 2022 · 8,965 citations
- Adding Conditional Control to Text-to-Image Diffusion ModelsLvmin Zhang, Anyi Rao, Maneesh AgrawalaICCV 2023 · 6,759 citations
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
- NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view ReconstructionPeng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt et al.NeurIPS 2021 · 2,500 citations
Related papers
- Instruct-NeRF2NeRF: Editing 3D Scenes with InstructionsAyaan Haque, Matthew Tancik, Alexei A. Efros, Aleksander Holynski et al.ICCV 2023 · 544 citations
- VDFE: Difference-Aware 3D Scene Editing with Non-Intrusive Video Diffusion Priors for Multi-View Consistency and EfficiencyChao Zhang, Fang Liu, Shuo Li, Yang Liu et al.CVPR 2026
- Denoising Diffusion via Image-Based RenderingTitas Anciukevicius, Fabian Manhardt, Federico Tombari, Paul HendersonICLR 2024 · 19 citations
- VisFusion: Visibility-Aware Online 3D Scene Reconstruction from VideosHuiyu Gao, Wei Mao, Miaomiao LiuCVPR 2023
- NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion ModelsSeung Wook Kim, Bradley Brown, Kangxue Yin, Karsten Kreis et al.CVPR 2023
