Lune

CVPR Findings2026

IRL-VLA: Vision-Language-Action Training via Reward World Model

Anqing Jiang, Gao Yu, Heng Yuwen, Yiru Wang, Wang Shuo, Jiang Hao, Sun Hao

Abstract

Vision-Language-Action (VLA) models hold great promise for autonomous driving, yet their practical deployment is hindered by two fundamental limitations. First, existing VLA architectures rely on open-loop imitation learning, which merely replicates behaviors from training data without reasoning about long-term consequences in diverse driving scenarios, leading to brittle and suboptimal policies. Second, effective closed-loop training typically requires high-fidelity sensor simulation, where domain gaps and computational overhead severely limit scalability. To address these challenges, we propose IRL-VLA, a novel closed-loop reinforcement learning framework comprising two core components: (i) a Reward World Model (RWM) learned via Inverse Reinforcement Learning (IRL) from expert driving demonstrations, which captures the underlying cost structure of human driving and enables simulation-free reward estimation; and (ii) a new policy optimization algorithm, Group Scoring-Filtered Policy Optimization (GSPO), that leverages RWM-provided rewards to jointly optimize safety, comfort, and traffic efficiency by selectively reinforcing high-quality trajectory proposals. Extensive experiments on the NAVSIM and Bench2Drive benchmarks show that IRL-VLA achieves state-of-the-art performance. Our model achieves 49.2 EPDMS in the NAVSIMV2 Navhard benchmark which demonstrates that high-performance, closed-loop VLA learning can be realized without reliance on expensive simulators.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Training Diffusion Models with Reinforcement Learning cites Planning with Diffusion for Flexible Behavior Synthesis
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on19

  1. 3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George DrettakisSIGGRAPH 2023 · 5,687 citations
  2. Planning with Diffusion for Flexible Behavior SynthesisMichael Janner, Yilun Du, Joshua B. Tenenbaum, Sergey LevineICML 2022 · 1,115 citations
  3. Training Diffusion Models with Reinforcement LearningKevin Black, Michael Janner, Yilun Du, Ilya Kostrikov et al.ICLR 2024 · 816 citations
  4. VAD: Vectorized Scene Representation for Efficient Autonomous DrivingBo Jiang, Shaoyu Chen, Qing Xu, Bencheng Liao et al.ICCV 2023 · 602 citations
  5. Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong BaselinePenghao Wu, Xiaosong Jia, Li Chen, Junchi Yan et al.NeurIPS 2022 · 444 citations

Related papers

  1. Driving with Advice: Large Model as Motion Advisor for Joint PlanningJunyin Wang, Jinlei Yu, Hao Lin, Huikai Liu et al.AAAI 2026
  2. Unleashing VLA Potentials in Autonomous Driving via Explicit Learning from FailuresYuechen Luo, Fang Li, Qimao Chen, Shaoqing Xu et al.CVPR 2026 · 21 citations
  3. AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-TuningZewei Zhou, Tianhui Cai, Seth Z. Zhao, Yun Zhang et al.NeurIPS 2025 · 310 citations
  4. Learning Vision-Language-Action World Models for Autonomous DrivingGuoqing Wang, Pin Tang, Xiangxuan Ren, Guodongfang Zhao, Bailan Feng, Chao MaCVPR Findings 2026
  5. WMPO: World Model-based Policy Optimization for Vision-Language-Action ModelsFangqi Zhu, Zhengyang Yan, Zicong Hong, Quanxin Shou et al.ICLR 2026 · 64 citations
Dusk over the sea between two cliffs drawn in fine vertical lines