GEM: Generating LiDAR World Model via Deformable Mamba
Yang Wu, Zhaojiang Liu, Qiang Meng, Youquan Liu, Renliang Weng, Jianjun Qian, Jian Yang, Jin Xie
Abstract
World models, which simulate environmental dynamics and generate sensor observations, are gaining increasing attention in autonomous driving. However, progress in LiDAR-based world models has lagged behind those built on camera videos or occupancy data, primarily due to two core challenges: the inherent disorder of LiDAR point clouds and the difficulty of distinguishing dynamic objects from static structures. To address these issues, we propose GEM: a Generative LiDAR world model that leverages deformable mamba architecture, significantly improving fidelity and imaginative capability. Specifically, leveraging the structural similarity between sequential laser scanning and Mamba's processing mechanism, we first tokenize LiDAR sweeps into compact representations via a custom LiDAR scene tokenizer. After unsupervised disentanglement of tokenized features via a dynamic-static separator, a tri-path deformable Mamba is introduced to perform selective scanning and adaptive gating fusion over the disentangled features, leading to enhanced spatial-temporal understanding of the world evolution. Optionally, a planner and a BEV layout controller can be integrated to explore the model's capability for autonomous rollout and its potential to generate ``what-if"scenarios. Extensive experiments show that GEM achieves state-of-the-art performances across diverse benchmarks and evaluation settings, demonstrating its superiority and effectiveness. Project page: https://github.com/wuyang98/GEM.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext a3abf646-dc8c-46b2-a603-d061f79f7d36Builds on25
- Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Scalable Diffusion Models with TransformersWilliam Peebles, Saining XieICCV 2023 · 5,568 citations
- Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space ModelLianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang et al.ICML 2024 · 1,725 citations
- Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-FreeZihan Qiu, Zekun Wang, Bo Zheng, Zeyu Huang et al.NeurIPS 2025 · 336 citations
Related papers
- BeyondMix: Leveraging Structural Priors and Long-Range Dependencies for Domain-Invariant LiDAR SegmentationYujia Chen, Rui Sun, Wangkai Li, Huayu Mai et al.NeurIPS 2025 · 8 citations
- DriveLiDAR4D: Sequential and Controllable LiDAR Scene Generation for Autonomous DrivingKaiwen Cai, Xinze Liu, Xia Zhou, Hengtong Hu et al.AAAI 2026
- La La LiDAR: Large-Scale Layout Generation from LiDAR DataYouquan Liu, Lingdong Kong, Weidong Yang, Xin Li et al.AAAI 2026 · 10 citations
- LiDARCrafter: Dynamic 4D World Modeling from LiDAR SequencesAlan Liang, Youquan Liu, Yu Yang, Dongyue Lu et al.AAAI 2026 · 12 citations
- LiDARsim: Realistic LiDAR Simulation by Leveraging the Real WorldSivabalan Manivasagam, Shenlong Wang, Kelvin Wong, Wenyuan Zeng et al.CVPR 2020
