Generative Omnimatte: Learning to Decompose Video into Layers

Input video Output: Omnimatte layers Object removal Layer editing See-through foreground Motion retiming Layer resizing + background replacement ActionShot (duplicating + retiming) Figure 1 . Generative Omnimatte. Our method decomposes a video into a set of RGBA omnimatte layers, where each layer consists of a fully-visible object and its associated effects like shadows and reflections. We improve upon existing work by adding a generative video prior, allowing our method to complete occluded regions (top, middle) and handle dynamic backgrounds (bottom).