Unsupervised Discovery of Object Radiance Fields

We study the problem of inferring an object-centric scene representation from a single image, aiming to derive a representation that is learned without supervision, explains the image formation process, and captures the scene's 3D nature. Most existing methods on scene decomposition lack one or more of these characteristics, due to the fundamental challenge in integrating powerful unsupervised inference schemes like deep networks with the complex 3D-to-2D image formation process. In this paper, we propose unsupervised discovery of Object Radiance Fields (uORF), integrating recent progresses in neural 3D scene representations and rendering with deep inference networks for unsupervised 3D scene decomposition. Trained on only multi-view RGB images, uORF learns to decompose complex scenes with diverse, textured background from a single image. We show that uORF enables novel tasks, such as scene segmentation and editing in 3D, and it performs well on these tasks and on novel view synthesis on three datasets * .