MACARONS: Mapping and Coverage Anticipation with RGB Online Self-Supervision

https://imagine.enpc.fr/ ˜guedona/MACARONS/ (a) NBV methods with a depth sensor (e.g., [27]) (b) Our approach MACARONS with an RGB sensor Figure 1. Mapping and Coverage Anticipation with RGB Online Self-Supervision. (a) NBV methods such as [27] rely on a depth sensor to perform path planning (bottom) and scan the environment (top). They need to be trained with explicit 3D supervision, generally on small-scale meshes. (b) Our approach MACARONS instead simultaneously learns to efficiently explore the scene and to reconstruct it (top) using an RGB sensor only. Its self-supervised, online learning process scales to large-scale and complex scenes.