SPSG: Self-Supervised Photometric Scene Generation From RGB-D Scans
Angela Dai, Yawar Siddiqui, Justus Thies, Julien Valentin, Matthias Nießner
Abstract
Figure 1 : Our SPSG approach formulates the problem of generating a complete, colored 3D model from an incomplete scan observation to be self-supervised, enabling training on incomplete real-world scan data. Our key idea is to leverage a 2D view-guided synthesis for self-supervision, comparing rendered views of our predicted model to the original RGB-D frames of the scan. Our 2D view-guided synthesis enables outperforming methods relying fully on 3D-based (self-)supervision.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 088cd0c2-4d3f-49f2-8b29-12947a57550fCited by top-tier papers18
- Neural RGB-D Surface ReconstructionDejan Azinovic, Ricardo Martin-Brualla, Dan B. Goldman, Matthias Nießner et al.CVPR 2022 · 272 citations
- Text2Tex: Text-driven Texture Synthesis via Diffusion ModelsDave Zhenyu Chen, Yawar Siddiqui, Hsin-Ying Lee, Sergey Tulyakov et al.ICCV 2023 · 262 citations
- 4DComplete: Non-Rigid Motion Estimation Beyond the Observable SurfaceYang Li, Hikari Takehara, Takafumi Taketomi, Bo Zheng et al.ICCV 2021 · 160 citations
- DiffComplete: Diffusion-based Generative 3D Shape CompletionRuihang Chu, Enze Xie, Shentong Mo, Zhenguo Li et al.NeurIPS 2023 · 66 citations
- Reference-guided Controllable Inpainting of Neural Radiance FieldsAshkan Mirzaei, Tristan Aumentado-Armstrong, Marcus A. Brubaker, Jonathan Kelly et al.ICCV 2023 · 49 citations
Builds on5
- PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human DigitizationShunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima et al.ICCV 2019 · 1,411 citations
- PointFlow: 3D Point Cloud Generation With Continuous Normalizing FlowsGuandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu et al.ICCV 2019 · 794 citations
- Texture Fields: Learning Texture Representations in Function SpaceMichael Oechsle, Lars M. Mescheder, Michael Niemeyer, Thilo Strauss et al.ICCV 2019 · 334 citations
- Adversarial Texture Optimization From RGB-D ScansJingwei Huang, Justus Thies, Angela Dai, Abhijit Kundu et al.CVPR 2020
- SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D ScansAngela Dai, Christian Diller, Matthias NießnerCVPR 2020
Related papers
- Unpaired Point Cloud Completion on Real Scans using Adversarial TrainingXuelin Chen, Baoquan Chen, Niloy J. MitraICLR 2020 · 146 citations
- Complete Structure Guided Point Cloud Completion via Cluster- and Instance-Level Contrastive LearningYang Chen, Yirun Zhou, Weizhong Zhang, Cheng JinNeurIPS 2025
- Towards Part-Based Understanding of RGB-D ScansAlexey Bokhovkin, Vladislav Ishimtsev, Emil Bogomolov, Denis Zorin et al.CVPR 2021
- P2C: Self-Supervised Point Cloud Completion from Single Partial CloudsRuikai Cui, Shi Qiu, Saeed Anwar, Jiawei Liu et al.ICCV 2023 · 40 citations
- DeOcc-1-to-3: 3D De-Occlusion from a Single Image via Self-Supervised Multi-View DiffusionYansong Qu, Shaohui Dai, Xinyang Li, Yuze Wang et al.AAAI 2026
