Slice3D: Multi-Slice, Occlusion-Revealing, Single View 3D Reconstruction

We introduce multi-slice reasoning, a new notion for single-view 3D reconstruction which challenges the current and prevailing belief that multi-view synthesis is the most natural conduit between single-view and 3D. Our key ob-servation is that object slicing is a more direct, and hence more advantageous, means to reveal occluded structures than altering camera views. Specifically, slicing can peel through any occluder without obstruction, and in the limit (i.e., with infinitely many slices), it is guaranteed to unveil all hidden object parts. We realize our idea by developing Slice3D, a novel method for single-view 3D reconstruction which first predicts multi-slice images from a single RGB input image and then integrates the slices into a 3D model using a coordinate-based transformer network to product a signed distance function. The slice images can be regressed or generated, both through a U-Net based network. For the former, we inject a learnable slice indicator code to desig-nate each decoded image into a spatial slice location, while the slice generator is a denoising diffusion model operating on the entirety of slice images stacked on the input channels. We conduct extensive evaluation against state-of-the-art alternatives to demonstrate superiority of our method, especially in recovering complex and severely occluded shape structures, amid ambiguities. All Slice3D results were produced by networks trained on a single Nvidia A40 GPU, with an inference time of less than 20 seconds.