Amodal Scene Analysis via Holistic Occlusion Relation Inference and Generative Mask Completion

Amodal scene analysis entails interpreting the occlusion relationship among scene elements and inferring the possible shapes of the invisible parts. Existing methods typically frame this task as an extended instance segmentation or a pair-wise object de-occlusion problem. In this work, we propose a new framework, which comprises a Holistic Occlusion Relation Inference (HORI) module followed by an instancelevel Generative Mask Completion (GMC) module. Unlike previous approaches, which rely on mask completion results for occlusion reasoning, our HORI module directly predicts an occlusion relation matrix in a single pass. This approach is much more efficient than the pair-wise de-occlusion process and it naturally handles mutual occlusion, a common but often neglected situation. Moreover, we formulate the mask completion task as a generative process and use a diffusion-based GMC module for instance-level mask completion. This improves mask completion quality and provides multiple plausible solutions. We further introduce a largescale amodal segmentation dataset which consists of highquality human annotations for amodal masks and occlusion relations, including mutual occlusions. Experiments on the newly proposed dataset and two public benchmarks demonstrate the advantages of our method on both efficient occlusion reasoning and plausible amodal mask completion. code public available at https://github.com/zbwxp/Amodal-AAAI.