MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation

Total3D InstPIFu SSR DiffCAD Gen3DSR REPARO Ours (a) Scene generation (b) Generalization Synthetic Data Real-world Image Stylized Image Figure 1. MIDI generates compositional 3D scenes from a single image by extending pre-trained image-to-3D object generation models to multi-instance diffusion models, incorporating a novel multi-instance attention mechanism that captures inter-object interactions. (a) shows our generated scenes compared with those reconstructed by existing methods. (b) presents our generated results on synthetic data, real-world images, and stylized images.