SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans

Figure 1: Our method takes as input a partial RGB-D scan and predicts a high-resolution 3D reconstruction while predicting unseen, missing geometry. Key to our approach is its self-supervised formulation, enabling training solely on real-world, incomplete scans. This not only obviates the need for synthetic ground truth, but is also capable of generating more complete scenes than any single target scene seen during training. To achieve high-quality surfaces, we further propose a new sparse generative neural network, capable of generating large-scale scenes at much higher resolution than existing techniques.