Lune

AAAI2026Top-tier venue

Collaboratively "Copy & Paste" 2D-3D Features for Complex Video-to-Video Motion Editing

Jia-Xing Zhong, Shijie Zhao, Junlin Li, Li Zhang

Abstract

Video-to-video human motion editing aims to transfer motion from a driving video to a reference video while preserving the background dynamics and the protagonist's original appearance. We identify critical limitations in existing methods that fail to capture the full complexity of human motions, particularly regarding: 1) location changes, 2) orientation variations, and 3) complicated non-upright poses. To address these challenges, we propose a framework that collaboratively "copies and pastes" 2D and 3D features across spatio-temporal dimensions into a shared representation space for motion guidance. Our approach achieves this through: 1) a mutual distillation mechanism that enhances the robustness and capability of individual encoders, and 2) a selective fusion module that adaptively weights and combines complementary information from spatio-temporal representations. To evaluate motion editing algorithms under challenging scenarios, we introduce a comprehensive benchmark dataset comprising real-world video clips from artistic gymnastics and figure skating competitions. These sports disciplines naturally encompass the three aforementioned aspects of motion complexity. Extensive experiments demonstrate that our approach significantly outperforms existing methods, particularly in handling intricate human motions.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • ControlVideo: Training-free Controllable Text-to-video Generation cites High-Resolution Image Synthesis with Latent Diffusion Models
  • ControlVideo: Training-free Controllable Text-to-video Generation cites Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
  • FateZero: Fusing Attentions for Zero-shot Text-based Video Editing cites High-Resolution Image Synthesis with Latent Diffusion Models
  • FateZero: Fusing Attentions for Zero-shot Text-based Video Editing cites Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
  • AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning cites High-Resolution Image Synthesis with Latent Diffusion Models
  • AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning cites Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
  • Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation cites High-Resolution Image Synthesis with Latent Diffusion Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on16

  1. High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
  2. AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific TuningYuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang et al.ICLR 2024 · 1,493 citations
  3. Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video GenerationJay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei et al.ICCV 2023 · 1,113 citations
  4. FateZero: Fusing Attentions for Zero-shot Text-based Video EditingChenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei et al.ICCV 2023 · 510 citations
  5. ControlVideo: Training-free Controllable Text-to-video GenerationYabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang et al.ICLR 2024 · 359 citations

Related papers

  1. Generative Video Motion Editing with 3D Point TracksYao-Chih Lee, Zhoutong Zhang, Jiahui Huang, Jui-Hsien Wang et al.CVPR 2026 · 23 citations
  2. PTUS: Photo-Realistic Talking Upper-Body Synthesis via 3D-Aware Motion Decomposition WarpingLuoyang Lin, Zutao Jiang, Xiaodan Liang, Liqian Ma et al.AAAI 2024 · 3 citations
  3. RealisMotion: Decomposed Human Motion Control and Video Generation in the World SpaceJingyun Liang, Jingkai Zhou, Shikai Li, Chenjie Cao et al.ICML 2026 · 9 citations
  4. REMOT: A Region-to-Whole Framework for Realistic Human Motion TransferQuanwei Yang, Xinchen Liu, Wu Liu, Hongtao Xie et al.ACM MM 2022 · 5 citations
  5. Motion-2-To-3: Leveraging 2D Motion Data for 3D Motion GenerationsRuoxi Guo, Huaijin Pi, Zehong Shen, Qing Shuai et al.ICCV 2025 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines