Video Motion Transfer with Diffusion Transformers

A lion walking through a busy market Robot walking on a sidewalk DiT Motorbiker driving around moonlit sand dunes Astronaut walking on the moon DiT Figure 1. Overview of DiTFlow. We propose a motion transfer method tailored for video Diffusion Transformers (DiT). We exploit a training-free strategy to transfer the motion of a reference video (top) to newly synthesized video content with arbitrary prompts (bottom). By optimizing DiT-specific positional embeddings, we can also synthesize new videos in a zero-shot manner.