Physics-Guided Motion Loss for Video Generation Model
Bowen Xue, Giuseppe Guarnera, Shuang Zhao, Zahra Montazeri
摘要
Current video diffusion models generate visually compelling content but often struggle with physical motion, producing subtle artifacts like rubber-sheet deformations and inconsistent object motion. We introduce a frequency-domain physics prior that improves motion plausibility without modifying model architectures. Our method decomposes common motion patterns (translation, rotation, scaling) into lightweight spectral losses. Applied to Open-Sora, MVDIT, and Hunyuan, our approach improves both motion accuracy and action recognition by ∼11% on average on OpenVID-1M (relative), while maintaining visual quality. Additional results on Wan 2.1-14B show consistent gains on video-quality and physics-oriented metrics. User studies show 74-83% preference for our physics-enhanced videos. It also reduces warping error by 22-37% (depending on the backbone) and improves temporal consistency scores. These results indicate that simple, global spectral cues are an effective drop-in regularizer for physically plausible motion in video diffusion.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper11
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific TuningYuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang 等ICLR 2024 · 被引用 1,493 次
- Flexible Diffusion Modeling of Long VideosWilliam Harvey, Saeid Naderiparizi, Vaden Masrani, Christian Weilbach 等NeurIPS 2022 · 被引用 384 次
- Make-A-Video: Text-to-Video Generation without Text-Video DataUriel Singer, Adam Polyak, Thomas Hayes, Xi Yin 等ICLR 2023 · 被引用 313 次
- Scaling Autoregressive Video ModelsDirk Weissenborn, Oscar Täckström, Jakob UszkoreitICLR 2020 · 被引用 252 次
相关 Paper
- MoAlign: Motion-Centric Representation Alignment for Video Diffusion ModelsAritra Bhowmik, Denis Korzhenkov, Cees G. M. Snoek, Amir Habibian 等ICLR 2026 · 被引用 15 次
- Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped NoiseRyan D. Burgert, Yuancheng Xu, Wenqi Xian, Oliver Pilarski 等CVPR 2025
- PhysDiff-VTON: Cross-Domain Physics Modeling and Trajectory Optimization for Virtual Try-OnShibin Mei, Bingbing NiNeurIPS 2025 · 被引用 4 次
- MotionRAG: Motion Retrieval-Augmented Image-to-Video GenerationChenhui Zhu, Yilu Wu, Shuai Wang, Gangshan Wu 等NeurIPS 2025 · 被引用 8 次
- SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction ScenariosLingwei Dang, Ruizhi Shao, Hongwen Zhang, Wei Min 等NeurIPS 2025 · 被引用 12 次
