Lune

CVPR2021Top-tier venue

Unsupervised 3D Shape Completion Through GAN Inversion

Junzhe Zhang, Xinyi Chen, Zhongang Cai, Liang Pan, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Bo Dai, Chen Change Loy

Abstract

Most 3D shape completion approaches rely heavily on partial-complete shape pairs and learn in a fully supervised manner. Despite their impressive performances on in-domain data, when generalizing to partial shapes in other forms or real-world partial scans, they often obtain unsatisfactory results due to domain gaps. In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time. ShapeInversion uses a GAN pre-trained on complete shapes by searching for a latent code that gives a complete shape that best reconstructs the given partial input. In this way, ShapeInversion no longer needs paired training data, and is capable of incorporating the rich prior captured in a well-trained generative model. On the ShapeNet benchmark, the proposed ShapeInversion outperforms the SOTA unsupervised method, and is comparable with supervised methods that are learned using paired data. It also demonstrates remarkable generalization ability, giving robust results for real-world scans and partial inputs of various forms and incompleteness levels. Importantly, ShapeInversion naturally enables a series of additional abilities thanks to the involvement of a pre-trained GAN, such as producing multiple valid complete shapes for an ambiguous partial input, as well as shape manipulation and interpolation.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

33 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Image Processing Using Multi-Code GAN Prior cites Seeing What a GAN Cannot Generate
  • Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion cites PU-GAN: A Point Cloud Upsampling Adversarial Network
  • Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion cites Morphing and Sampling Network for Dense Point Cloud Completion
  • Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion cites Unpaired Point Cloud Completion on Real Scans using Adversarial Training
  • HoloPart: Generative 3D Part Amodal Segmentation cites Unpaired Point Cloud Completion on Real Scans using Adversarial Training
  • HoloPart: Generative 3D Part Amodal Segmentation cites DiffComplete: Diffusion-based Generative 3D Shape Completion
  • A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion cites Morphing and Sampling Network for Dense Point Cloud Completion
  • DiffComplete: Diffusion-based Generative 3D Shape Completion cites Unpaired Point Cloud Completion on Real Scans using Adversarial Training
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers33

  1. A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud CompletionZhaoyang Lyu, Zhifeng Kong, Xudong Xu, Liang Pan et al.ICLR 2022 · 159 citations
  2. ShapeFormer: Transformer-based Shape Completion via Sparse RepresentationXingguang Yan, Liqiang Lin, Niloy J. Mitra, Dani Lischinski et al.CVPR 2022 · 124 citations
  3. Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud CompletionTong Wu, Liang Pan, Junzhe Zhang, Tai Wang et al.NeurIPS 2021 · 104 citations
  4. DiffComplete: Diffusion-based Generative 3D Shape CompletionRuihang Chu, Enze Xie, Shentong Mo, Zhenguo Li et al.NeurIPS 2023 · 66 citations
  5. HoloPart: Generative 3D Part Amodal SegmentationYunhan Yang, Yuanchen Guo, Yukun Huang, Zi-Xin Zou et al.ICLR 2026 · 62 citations

Builds on8

  1. PU-GAN: A Point Cloud Upsampling Adversarial NetworkRuihui Li, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or et al.ICCV 2019 · 496 citations
  2. Morphing and Sampling Network for Dense Point Cloud CompletionMinghua Liu, Lu Sheng, Sheng Yang, Jing Shao et al.AAAI 2020 · 363 citations
  3. Seeing What a GAN Cannot GenerateDavid Bau, Jun-Yan Zhu, Jonas Wulff, William S. Peebles et al.ICCV 2019 · 342 citations
  4. Unpaired Point Cloud Completion on Real Scans using Adversarial TrainingXuelin Chen, Baoquan Chen, Niloy J. MitraICLR 2020 · 146 citations
  5. Image Processing Using Multi-Code GAN PriorJinjin Gu, Yujun Shen, Bolei ZhouCVPR 2020

Related papers

  1. Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN InversionYushi Lan, Xuyi Meng, Shuai Yang, Chen Change Loy et al.CVPR 2023
  2. Denoise and Contrast for Category Agnostic Shape CompletionAntonio Alliegro, Diego Valsesia, Giulia Fracastoro, Enrico Magli et al.CVPR 2021
  3. Learning a Structured Latent Space for Unsupervised Point Cloud CompletionYingjie Cai, Kwan-Yee Lin, Chao Zhang, Qiang Wang et al.CVPR 2022 · 46 citations
  4. Reverse2Complete: Unpaired Multimodal Point Cloud Completion via Guided DiffusionWenxiao Zhang, Hossein Rahmani, Xun Yang, Jun LiuACM MM 2024 · 4 citations
  5. Shape Self-Correction for Unsupervised Point Cloud UnderstandingYe Chen, Jinxian Liu, Bingbing Ni, Hang Wang et al.ICCV 2021 · 58 citations
Dusk over the sea between two cliffs drawn in fine vertical lines