-World: Intra-Inter Tokenization for Efficient Dynamic 4D Scene Forecasting
Zhimin Liao, Ping Wei, Ruijie Zhang, Shuaijia Chen, Haoxuan Wang, Ziyang Ren
摘要
Forecasting the evolution of 3D scenes and generating unseen scenarios via occupancy-based world models offer substantial potential for addressing corner cases in autonomous driving systems. While tokenization has revolutionized image and video generation, efficiently tokenizing complex 3D scenes remains a critical challenge for 3D world models. To address this issue, we propose -World, an efficient framework for 4D occupancy forecasting. Our method decouples scene tokenization into intra-scene and inter-scene tokenizers. The intra-scene tokenizer employs a multi-scale residual quantization strategy to hierarchically compress scenes while preserving spatial details. The inter-scene tokenizer residually aggregates temporal dependencies across timesteps. This dual design preserves the compactness of 3D tokenizers while retaining the dynamic expressiveness of 4D tokenizers. Unlike decoder-only GPT-style autoregressive models, -World adopts an encoder-decoder architecture. The encoder aggregates spatial context from the current scene and predicts a transformation matrix to enable high-level control over scene generation. The decoder, conditioned on transformation matrix and historical tokens, ensures temporal consistency during generation. Experiments demonstrate that -World achieves state-of-theart performance, outperforming existing methods by in and 36.9% in for 4D occupancy forecasting while exhibiting exceptional computational efficiency. It nearly requires 2.9 GB of training memory and achieves real-time inference at 37.0 FPS. Our code is available on https://github.com/lzzzzzm/II-World.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper2
- SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World ModelJiayuan Du, Yiming Zhao, Zhenglong Guo, Yong Pan 等CVPR 2026 · 被引用 6 次
- The Structure-Equivalent Prior: Unifying Temporal Dynamics and 3D Evolution in 4D Latent SpaceJingyuan Gao, Tianyu Shen, Ruosen Hao, Te Guo 等AAAI 2026
它引用的顶会 Paper28
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser 等CVPR 2022 · 被引用 13,123 次
- Deformable DETR: Deformable Transformers for End-to-End Object DetectionXizhou Zhu, Weijie Su, Lewei Lu, Bin Li 等ICLR 2021 · 被引用 7,353 次
- Scalable Diffusion Models with TransformersWilliam Peebles, Saining XieICCV 2023 · 被引用 5,568 次
- SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR SequencesJens Behley, Martin Garbade, Andres Milioto, Jan Quenzel 等ICCV 2019 · 被引用 2,345 次
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale PredictionKeyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng 等NeurIPS 2024 · 被引用 1,199 次
相关 Paper
- SparseWorld: A Flexible, Adaptive, and Efficient 4D Occupancy World Model Powered by Sparse and Dynamic QueriesChenxu Dang, Haiyan Liu, Jason Bao, Pei An 等AAAI 2026 · 被引用 6 次
- Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous DrivingYu Yang, Jianbiao Mei, Yukai Ma, Siliang Du 等AAAI 2025 · 被引用 53 次
- Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete DiffusionLunjun Zhang, Yuwen Xiong, Ze Yang, Sergio Casas 等ICLR 2024 · 被引用 105 次
- DIO: Decomposable Implicit 4D Occupancy-Flow World ModelChristopher Diehl, Quinlan Sykora, Ben Agro, Thomas Gilles 等CVPR 2025
- Overcoming Challenges of Long-Horizon Prediction in Driving World ModelsArian Mousakhan, Sudhanshu Mittal, Silvio Galesso, Karim Farid 等NeurIPS 2025 · 被引用 1 次
