Omnimatte3D: Associating Objects and Their Effects in Unconstrained Monocular Video

Columbia 2 Vector Institute for AI 3 Canada CIFAR AI Chair 4 Google Input RGB Input Masks Input Depth Layer 1 + Background Layer 2 + Background Figure 1. Layer decomposition under strong camera parallax. Given an input video with unconstrained camera motion and approximate object masks and depth (left), our method estimates a layered representation composed of a background layer and object layers containing the subjects of interest and their associated effects (e.g. shadows). Results of combining object layers and background are shown on right.