CamGeo: Sparse Camera-Conditioned Image-to-Video Generation with 3D Geometry Priors
Xuanyi Liu, Deyi Ji, Liqun Liu, Lanyun Zhu, Xuhang Chen, Qianxiong Xu, Peng Shu, Huan Yu, Jie Jiang, Feng Gao, Siwei Ma
摘要
Sparse camera-conditioned image-to-video generation presents a pivotal challenge: synthesizing geometrically consistent 3D motion from minimal pose cues. Existing methods, which largely rely on dense supervision or naive interpolation, suffer from severe pose drift and motion discontinuities due to the lack of robust 3D priors. In this paper, we introduce CamGeo, a novel framework that distills rich 3D geometric knowledge from a pre-trained video-to-3D model (VGGT) directly into the diffusion backbone. To achieve this without incurring inference latency, we propose a training-only distillation strategy. Specifically, CamGeo incorporates: (1) keyframe trajectory distillation that enforces cycle-consistency with sparse input poses, (2) cross-frame consistency distillation with both camera trajectory and depth constraints to generate consistent structure across unsupervised frames, and (3) a three-stage coarse-to-fine curriculum learning, progressively scales geometric complexity, from global structure coherence to fine-grained refinement, achieving stable optimization. Extensive experiments demonstrate that CamGeo achieves consistent improvements under various sparsity ratios.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper1
问问它们各自怎么用它它引用的顶会 Paper13
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific TuningYuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang 等ICLR 2024 · 被引用 1,493 次
- ScanNet++: A High-Fidelity Dataset of 3D Indoor ScenesChandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, Angela DaiICCV 2023 · 被引用 659 次
- Light Field Networks: Neural Scene Representations with Single-Evaluation RenderingVincent Sitzmann, Semon Rezchikov, Bill Freeman, Josh Tenenbaum 等NeurIPS 2021 · 被引用 426 次
- MotionCtrl: A Unified and Flexible Motion Controller for Video GenerationZhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li 等SIGGRAPH 2024 · 被引用 123 次
相关 Paper
- TrajVG: 3D Trajectory-Coupled Visual Geometry LearningXingyu Miao, Weiguang Zhao, Tao Lu, Linning Xu 等SIGGRAPH 2026
- UniScene-MoTion: Unified Scene & Motion-aware Diffusion Transition FrameworkRui Jiang, Chongmian Wang, Xinghe Fu, Yehao Lu 等AAAI 2026
- CameraNoise: Enabling Faithful Camera Control in Video Diffusion through Geometry-Flow-Guided Noise WarpingHaoyu Zhao, Jiaxi Gu, Haoran Chen, Qingping Zheng 等ICML 2026
- ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video GenerationOmar El Khalifi, Thomas Rossi, Oscar Fossey, Thibault Fouque 等SIGGRAPH 2026
- GeoVideo: Introducing Geometric Regularization into Video Generation ModelYunpeng Bai, Shaoheng Fang, Chaohui Yu, Fan Wang 等NeurIPS 2025 · 被引用 18 次
