From Language to Driving: A Dual-Loop SLM-Enhanced Framework for Multi-Planner Scheduling via a Domain-Specific Language
Jiawei Liu, Xun Gong, Muli Yang, Xingrui Yu, Fen Fang, Xulei Yang, Ivor W. Tsang, Yunfeng Hu, Hong Chen, Qing Guo
Abstract
Advancing from usable to collaborative autonomy requires driving systems to execute passenger instructions safely and reliably. This work formulates instruction realization as scheduling across multiple motion planners and presents a dual-loop framework that provides a transparent decision chain from natural language to vehicle control. The outer loop uses a small language model (SLM) for high-level, low-frequency semantic reasoning and schedule generation, while the inner loop performs low-level, high-frequency schedule execution and vehicle control. To compensate for the SLM's limited capacity, the framework integrates receding-horizon scheduling to segment long-horizon instruction tasks, a domainspecific language (DSL) that restricts SLM outputs to a scheduling-oriented subspace, and reinforcement learning in high-fidelity urban traffic to refine the SLM's DSL proficiency and scheduling performance. Experiments show that the framework improves instructioncompletion rates while maintaining high safety and compliance relative to multiple baselines.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext e3b1cceb-9c88-4f8a-a68b-ac4401316e33Builds on16
- Understanding the Effects of RLHF on LLM Generalisation and DiversityRobert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina et al.ICLR 2024 · 332 citations
- AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-TuningZewei Zhou, Tianhui Cai, Seth Z. Zhao, Yun Zhang et al.NeurIPS 2025 · 310 citations
- DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language ModelsLicheng Wen, Daocheng Fu, Xin Li, Xinyu Cai et al.ICLR 2024 · 255 citations
- FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous DrivingShuang Zeng, Xinyuan Chang, Mengwei Xie, Xinran Liu et al.NeurIPS 2025 · 228 citations
- LMDrive: Closed-Loop End-to-End Driving with Large Language ModelsHao Shao, Yuxuan Hu, Letian Wang, Guanglu Song et al.CVPR 2024 · 114 citations
Related papers
- Open-Ended Instruction Realization with LLM-Enabled Multi-Planner Scheduling in Autonomous VehiclesJiawei Liu, Xun Gong, Fen Fang, Muli Yang et al.CVPR 2026
- Driving with Advice: Large Model as Motion Advisor for Joint PlanningJunyin Wang, Jinlei Yu, Hao Lin, Huikai Liu et al.AAAI 2026
- LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model ProgramsYunsheng Ma, Can Cui, Xu Cao, Wenqian Ye et al.CVPR 2024 · 39 citations
- Adadrive: Self-Adaptive Slow-Fast System for Language-Grounded Autonomous DrivingRuifei Zhang, Junlin Xie, Wei Zhang, Weikai Chen et al.ICCV 2025 · 3 citations
- Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement LearningYue Li, Meng Tian, Dechang Zhu, Jiangtong Zhu et al.AAAI 2026 · 27 citations
