MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency

mingyexu.github.io/mm3dscene Figure 1 . How to apply masked modeling for large-scale 3D scenes? (a) Conventional random masked modeling on 3D scenes may cause a high risk of uncertainty.In this figure, a chair and a TV are totally masked, which are extremely difficult to be recovered without any context guidance. (b) Our MM-3DScene exploits local statistics to discover and preserve representative structured points, effectively simplifying the pretext task. At each learning step, our method focuses on restoring regional geometry, and enjoys less ambiguity. Moreover, since unmasked areas are underexplored during reconstruction, the model is encouraged to maintain the intrinsic spatial consistency on unmasked points between different masking ratios, which requires the consistent understanding of unmasked areas.