Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Driving
Abstract
End-to-End Autonomous Driving (E2EAD) methods typically rely on supervised perception tasks to extract explicit scene information (e.g., objects, maps). This reliance necessitates expensive annotations and constrains deployment and data scalability in real-time applications. In this paper, we introduce SSR, a novel framework that utilizes only 16 navigation-guided tokens as Sparse Scene Representation, efficiently extracting crucial scene information for E2EAD. Our method eliminates the need for human-designed supervised sub-tasks, allowing computational resources to concentrate on essential elements directly related to navigation intent. We further introduce a temporal enhancement module, aligning predicted future scenes with actual future scenes through self-supervision. SSR achieves a 27.2% relative reduction in L2 error and a 51.6% decrease in collision rate to UniAD in nuScenes, with a 10.9× faster inference speed and 13× faster training time. Moreover, SSR outperforms VAD-Base with a 48.6point improvement on driving score in CARLA's Town05 Long benchmark. This framework represents a significant leap in real-time autonomous driving systems and paves the way for future scalable deployment. Code is available at https://github.com/PeidongLi/SSR.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
19 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline cites Exploring the Limitations of Behavior Cloning for Autonomous Driving
- Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution cites Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection
- Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution cites VAD: Vectorized Scene Representation for Efficient Autonomous Driving
- SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse World cites Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline
- SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse World cites Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution
- SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse World cites VAD: Vectorized Scene Representation for Efficient Autonomous Driving
- VAD: Vectorized Scene Representation for Efficient Autonomous Driving cites Exploring the Limitations of Behavior Cloning for Autonomous Driving
- VAD: Vectorized Scene Representation for Efficient Autonomous Driving cites VectorMapNet: End-to-end Vectorized HD Map Learning
- DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving cites Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline
- DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving cites VAD: Vectorized Scene Representation for Efficient Autonomous Driving
- ResWorld: Temporal Residual World Model for End-to-End Autonomous Driving cites Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection
- ResWorld: Temporal Residual World Model for End-to-End Autonomous Driving cites VAD: Vectorized Scene Representation for Efficient Autonomous Driving
- WorldRFT: Latent World Model Planning with Reinforcement Fine-Tuning for Autonomous Driving cites VAD: Vectorized Scene Representation for Efficient Autonomous Driving
Cited by top-tier papers19
- Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene EvolutionBozhou Zhang, Nan Song, Jingyu Li, Xiatian Zhu et al.NeurIPS 2025 · 29 citations
- ResWorld: Temporal Residual World Model for End-to-End Autonomous DrivingJinqing Zhang, Zehua Fu, Zelin Xu, Wenying Dai et al.ICLR 2026 · 13 citations
- SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse WorldJungho Kim, Jiyong Oh, Seunghoon Yu, Hongjae Shin et al.CVPR 2026 · 8 citations
- DynVLA: Learning World Dynamics for Action Reasoning in Autonomous DrivingShuyao Shang, Bing Zhan, Yunfei Yan, Yuqi Wang et al.ICML 2026 · 8 citations
- WorldRFT: Latent World Model Planning with Reinforcement Fine-Tuning for Autonomous DrivingPengxuan Yang, Ben Lu, Zhongpu Xia, Chao Han et al.AAAI 2026 · 8 citations
Builds on20
- Exploring the Limitations of Behavior Cloning for Autonomous DrivingFelipe Codevilla, Eder Santana, Antonio M. López, Adrien GaidonICCV 2019 · 666 citations
- VAD: Vectorized Scene Representation for Efficient Autonomous DrivingBo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao et al.ICCV 2023 · 602 citations
- Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong BaselinePenghao Wu, Xiaosong Jia, Li Chen, Junchi Yan et al.NeurIPS 2022 · 444 citations
- Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object DetectionShihao Wang, Yingfei Liu, Tiancai Wang, Ying Li et al.ICCV 2023 · 399 citations
- VectorMapNet: End-to-end Vectorized HD Map LearningYicheng Liu, Tianyuan Yuan, Yue Wang, Yilun Wang et al.ICML 2023 · 332 citations
Related papers
- DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous DrivingXiaosong Jia, Junqi You, Zhiyuan Zhang, Junchi YanICLR 2025
- Enhancing End-to-End Autonomous Driving with Latent World ModelYingyan Li, Lue Fan, Jiawei He, Yuqi Wang et al.ICLR 2025
- World4Drive: End-to-End Autonomous Driving via Intention-Aware Physical Latent World ModelYupeng Zheng, Pengxuan Yang, Zebin Xing, Qichao Zhang et al.ICCV 2025 · 16 citations
- Rethinking the Spatio-Temporal Alignment of End-to-End 3D PerceptionXiaoyu Li, Peidong Li, Xian Wu, Long Shi et al.AAAI 2026 · 1 citation
- Self-Supervised Simultaneous Multi-Step Prediction of Road Dynamics and Cost MapElmira Amirloo Abolfathi, Mohsen Rohani, Ershad Banijamali, Jun Luo et al.CVPR 2021
