PanoWorld-X: Generating Explorable Panoramic Worlds via Sphere-Aware Video Diffusion
Yuyang Yin, Hao-Xiang Guo, Fangfu Liu, Mengyu Wang, Hanwen Liang, Eric Li, Yikai Wang, Xiaojie Jin, Yao Zhao, Yunchao Wei
摘要
Achieving a complete and explorable 360-degree visual world is a cornerstone of immersive content creation. While recent advances in video generation have achieved impressive results, they follow a 2D paradigm that treats content generation as transitions of 2D pixels, lacking an intrinsic understanding of the physical 3D world, resulting in frequent geometric inconsistencies. To achieve an explorable and physical-consistent visual world, the generation process should shift to a 3D paradigm: the visual content is governed by the physical relationships of the entire 3D environment together with 3D motion signals. However, under this setting, the conventional modeling methods and control signals, such as spatial attention computation in a 2D space, become unsuitable and ineffective. To address this, we propose PanoWorld-X for explorable 3D scene video generation. Our framework is built on the panoramic representation, which naturally maps a 3D scene into a standard format and provides an ideal basis for consistency. Specifically, we first develop a data curation pipeline to produce high-quality and large-motion 3D scene evolution with movement trajectories. To achieve precise control, we design the Exploration Panoramic Plücker Embedding (PPE), a guidance signal tailored for 3D motion. Furthermore, leveraging the spherical geometric properties of panoramic data, we propose a sphere-aware attention mechanism, which can capture true geometric adjacency by reprojecting features onto a spherical surface. Extensive experiments demonstrate that PanoWorld-X achieves superior performance in motion range, control precision, and visual quality, underscoring its potential for real-world applications.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper1
问问它们各自怎么用它它引用的顶会 Paper29
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh 等ICML 2021 · 被引用 47,906 次
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 被引用 21,477 次
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser 等CVPR 2022 · 被引用 13,123 次
- Adding Conditional Control to Text-to-Image Diffusion ModelsLvmin Zhang, Anyi Rao, Maneesh AgrawalaICCV 2023 · 被引用 6,759 次
相关 Paper
- 360Explorer: Exploring 4D Controllable World in Panoramic VideosXinhua Cheng, Haiyang Zhou, Wangbo Yu, Tanghui Jia 等AAAI 2026
- ViewPoint: Panoramic Video Generation with Pretrained Diffusion ModelsZixun Fang, Kai Zhu, Zhiheng Liu, Yu Liu 等NeurIPS 2025 · 被引用 2 次
- PanFlow: Decoupled Motion Control for Panoramic Video GenerationCheng Zhang, Hanwen Liang, Donny Y. Chen, Qianyi Wu 等AAAI 2026 · 被引用 1 次
- OmniRoam: World Wandering via Long-Horizon Panoramic Video GenerationYuheng Liu, Xin Lin, Xinke Li, Baihan Yang 等SIGGRAPH 2026 · 被引用 1 次
- Beyond the Frame: Generating 360° Panoramic Videos from Perspective VideosRundong Luo, Matthew Wallingford, Ali Farhadi, Noah Snavely 等ICCV 2025 · 被引用 2 次
