UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Models
Tianxing Xu, Zi-Xuan Wang, Guangyuan Wang, Li Hu, Zhongyi Zhang, Peng Zhang, Bang Zhang, Song-Hai Zhang
Abstract
World models based on video generation demonstrate remarkable potential for simulating interactive environments yet suffer from persistent difficulties in two key areas: maintaining long-term content consistency when scenes are revisited and enabling precise camera control from user-specified inputs. Existing methods based on explicit 3D reconstruction often compromise flexibility in unbounded scenarios and struggle to preserve fine-grained structures. Alternative methods rely directly on previously generated frames without establishing explicit spatial correspondence, thereby limiting controllability and consistency. To address these limitations, we present UCM, a novel framework for unified modeling of long-term memory and precise camera control via a time-aware positional encoding warping mechanism. To reduce computational overhead, we design an efficient dual-stream diffusion transformer for high-fidelity generation. Moreover, we introduce a scalable data curation strategy that utilizes point-cloud-based rendering to simulate scene revisiting, enabling training on over 500K monocular videos. Extensive experiments on real-world and synthetic benchmarks demonstrate that UCM significantly outperforms state-of-the-art methods on long-term scene consistency, while achieving precise camera controllability in high-fidelity video generation. Our code is released at https://humanaigc.github.io/ucm-webpage/.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 2a5eadb0-24ee-4ed1-a305-feb5f76ec0ecBuilds on33
- Scalable Diffusion Models with TransformersWilliam Peebles, Saining XieICCV 2023 · 5,568 citations
- Diffusion Forcing: Next-token Prediction Meets Full-Sequence DiffusionBoyuan Chen, Diego Marti Monso, Yilun Du, Max Simchowitz et al.NeurIPS 2024 · 751 citations
- Depth Anything 3: Recovering the Visual Space from Any ViewsHaotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen et al.ICLR 2026 · 720 citations
- Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene UnderstandingMike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar et al.ICCV 2021 · 633 citations
- Self Forcing: Bridging the Train-Test Gap in Autoregressive Video DiffusionXun Huang, Zhengqi Li, Guande He, Mingyuan Zhou et al.NeurIPS 2025 · 628 citations
Related papers
- WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric MemoriesYisu Zhang, Chenjie Cao, Tengfei Wang, Xuhui Zuo et al.CVPR 2026 · 13 citations
- Latent-Reframe: Enabling Camera Control for Video Diffusion Models Without TrainingZhenghong Zhou, Jie An, Jiebo LuoICCV 2025 · 14 citations
- TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion ModelsMark Yu, Wenbo Hu, Jinbo Xing, Ying ShanICCV 2025 · 25 citations
- Unified Camera Positional Encoding for Controlled Video GenerationCheng Zhang, Boying Li, Meng Wei, Yan-Pei Cao et al.CVPR 2026 · 38 citations
- World-consistent Video Diffusion with Explicit 3D ModelingQihang Zhang, Shuangfei Zhai, Miguel Ángel Bautista Martin, Kevin Miao et al.CVPR 2025
