Generative Video Propagation

Original Frame #1 Edited Frame #1 Input Video Frames (top) / Propagated by Ours (bottom) (b) Background Replacement (d) Object & Effect Tracking (e) Multiple Edits (a) Object & Effect Removal (c) Object Insertion 1 2 3 3 Figure 1. GenProp. We propose a generative video propagation framework (GenProp), which can seamlessly propagate any first frame edit through the video. GenProp supports a wide range of video applications, including (a) complete object removal with effects such as shadows and reflections, (b) background replacement with realistic effects, (c) object insertion where inserted objects have physically plausible motion (i.e., blueberries falling while spoon goes up), (d) tracking of objects and their associated effects, and (e) multiple edits (outpainting, insertion, removal) at a single inference run.