SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Driving
Xuesong Chen, Linjiang Huang, Tao Ma, Rongyao Fang, Shaoshuai Shi, Hongsheng Li
Abstract
The integration of Vision-Language Models (VLMs) into autonomous driving systems has shown promise in addressing key challenges such as learning complexity, interpretability, and common-sense reasoning. However, existing approaches often struggle with efficient integration and realtime decision-making due to computational demands. In this paper, we introduce SOLVE, an innovative framework that synergizes VLMs with end-to-end (E2E) models to enhance autonomous vehicle planning. Our approach emphasizes knowledge sharing at the feature level through a shared visual encoder, enabling comprehensive interaction between VLM and E2E components. We propose a Trajectory Chain-of-Thought (T-CoT) paradigm, which progressively refines trajectory predictions, reducing uncertainty and improving accuracy. By employing a temporal decoupling strategy, SOLVE achieves efficient cooperation by aligning high-quality VLM outputs with E2E real-time performance. Evaluated on the nuScenes dataset, our method demonstrates significant improvements in trajectory prediction accuracy, paving the way for more robust and reliable autonomous driving systems.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
8 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-based Token Pruning cites Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset
- FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-based Token Pruning cites VAD: Vectorized Scene Representation for Efficient Autonomous Driving
- FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-based Token Pruning cites DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets
- SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse World cites VAD: Vectorized Scene Representation for Efficient Autonomous Driving
- DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets cites Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset
- ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving cites Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset
- ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving cites Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection
- ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving cites VAD: Vectorized Scene Representation for Efficient Autonomous Driving
- ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving cites DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets
- DriveAgent-R1: Advancing VLM-based Autonomous Driving with Active Perception and Hybrid Thinking cites VAD: Vectorized Scene Representation for Efficient Autonomous Driving
- SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving cites VAD: Vectorized Scene Representation for Efficient Autonomous Driving
Cited by top-tier papers8
- SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous DrivingPeizheng Li, Zhenghao Zhang, David Holtz, Hang Yu et al.CVPR 2026 · 32 citations
- FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-based Token PruningJiajun Cao, Qizhe Zhang, Peidong Jia, Xuhui Zhao et al.AAAI 2026 · 18 citations
- ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous DrivingQihang Peng, Xuesong Chen, Chenye Yang, Shaoshuai Shi et al.CVPR 2026 · 10 citations
- SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse WorldJungho Kim, Jiyong Oh, Seunghoon Yu, Hongjae Shin et al.CVPR 2026 · 8 citations
- DriveAgent-R1: Advancing VLM-based Autonomous Driving with Active Perception and Hybrid ThinkingWeicheng Zheng, Xiaofei Mao, Nanfei Ye, Pengxiang Li et al.ICLR 2026 · 8 citations
Builds on11
- Flamingo: a Visual Language Model for Few-Shot LearningJean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech et al.NeurIPS 2022 · 6,707 citations
- Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion DatasetScott Ettinger, Shuyang Cheng, Benjamin Caine, Chenxi Liu et al.ICCV 2021 · 817 citations
- VAD: Vectorized Scene Representation for Efficient Autonomous DrivingBo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao et al.ICCV 2023 · 602 citations
- DenseTNT: End-to-end Trajectory Prediction from Dense Goal SetsJunru Gu, Chen Sun, Hang ZhaoICCV 2021 · 563 citations
- Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object DetectionShihao Wang, Yingfei Liu, Tiancai Wang, Ying Li et al.ICCV 2023 · 399 citations
Related papers
- AutoDrive-P3: Unified Chain of Perception-Prediction-Planning Thought via Reinforcement Fine-TuningYuqi Ye, Zijian Zhang, Junhong Lin, Shangkun Sun et al.ICLR 2026 · 16 citations
- VLP: Vision Language Planning for Autonomous DrivingChenbin Pan, Burhaneddin Yaman, Tommaso Nesti, Abhirup Mallik et al.CVPR 2024 · 53 citations
- FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous DrivingShuang Zeng, Xinyuan Chang, Mengwei Xie, Xinran Liu et al.NeurIPS 2025 · 228 citations
- Orion: A Holistic End-To-End Autonomous Driving Framework by Vision-Language Instructed Action GenerationHaoyu Fu, Diankun Zhang, Zongchuang Zhao, Jianfeng Cui et al.ICCV 2025 · 19 citations
- AutoMoT: A Unified Vision-Language-Action Model with Asynchronous Mixture -of-Transformers for End-to-End Autonomous DrivingWenhui (Oscar) Huang, Songyan Zhang, Qihang Huang, Zhidong Wang et al.ICML 2026 · 6 citations
