SPSG: Self-Supervised Photometric Scene Generation From RGB-D Scans

Figure 1 : Our SPSG approach formulates the problem of generating a complete, colored 3D model from an incomplete scan observation to be self-supervised, enabling training on incomplete real-world scan data. Our key idea is to leverage a 2D view-guided synthesis for self-supervision, comparing rendered views of our predicted model to the original RGB-D frames of the scan. Our 2D view-guided synthesis enables outperforming methods relying fully on 3D-based (self-)supervision.