FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Prediction
Yixiang Dai, Fan Jiang, Chiyu Wang, Mu Xu, Yonggang Qi
摘要
High-quality 3D world models are pivotal for embodied intelligence and Artificial General Intelligence (AGI), underpinning applications such as AR/VR content creation and robotic navigation. Despite the established strong imaginative priors, current video foundation models lack explicit 3D grounding capabilities, thus being limited in both spatial consistency and their utility for downstream 3D reasoning tasks. In this work, we present FANTASYWORLD, a geometry-enhanced framework that augments frozen video foundation models with a trainable geometric branch, enabling joint modeling of video latents and an implicit 3D field in a single forward pass. Our approach introduces cross-branch supervision, where geometry cues guide video generation and video priors regularize 3D prediction, thus yielding consistent and generalizable 3D-aware video representations. Notably, the resulting latents from the geometric branch can potentially serve as versatile representations for downstream 3D tasks such as novel view synthesis and navigation, without requiring per-scene optimization or fine-tuning. Extensive experiments show that FANTASYWORLD effectively bridges video imagination and 3D perception, outperforming recent geometry-consistent baselines in multi-view coherence and style consistency. Ablation studies further confirm that these gains stem from the unified backbone and cross-branch information exchange. Figure 1: FANTASYWORLD overview. Given multimodal inputs (image, text, and camera trajectory), the model generates photorealistic videos along the specified views while constructing an implicit 3D representation for consistent geometry.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper2
- CineScene: Implicit 3D as Effective Scene Representation for Cinematic Video GenerationKaiyi Huang, Yukun Huang, Yu Li, Jianhong Bai 等CVPR 2026 · 被引用 7 次
- WorldReel: 4D Video Generation with Consistent Geometry and Motion ModelingShaoheng Fang, Hanwen Jiang, Yunpeng Bai, Niloy J. Mitra 等CVPR 2026 · 被引用 3 次
它引用的顶会 Paper28
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 被引用 5,687 次
- Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content CreationRui Chen, Yongwei Chen, Ningxin Jiao, Kui JiaICCV 2023 · 被引用 769 次
- DreamFusion: Text-to-3D using 2D DiffusionBen Poole, Ajay Jain, Jonathan T. Barron, Ben MildenhallICLR 2023 · 被引用 463 次
- DUSt3R: Geometric 3D Vision Made EasyShuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii 等CVPR 2024 · 被引用 302 次
相关 Paper
- Future Dynamic 3D Reconstruction: A 3D World Model with Disentangled Ego-MotionNils Morbitzer, Jonathan Evers, Artem Savkin, Thomas Stauner 等ICML 2026
- Towards Realistic and Consistent Orbital Video Generation via 3D Foundation PriorsRong Wang, Ruyi Zha, Ziang Cheng, Jiayu Yang 等CVPR 2026
- Gen3R: 3D Scene Generation Meets Feed-Forward ReconstructionJiaxin Huang, Yuanbo Yang, Bangbang Yang, Lin Ma 等CVPR 2026 · 被引用 24 次
- Video World Models with Long-term Spatial MemoryTong Wu, Shuai Yang, Ryan Po, Yinghao Xu 等NeurIPS 2025 · 被引用 145 次
- Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited ViewsZhangquan Chen, Manyuan Zhang, Xinlei Yu, Xufang Luo 等CVPR 2026 · 被引用 61 次
