ScenePilot: Controllable Boundary-Driven Critical Scenario Generation for Autonomous Driving
Qiyu Ruan, YUXUAN WANG, He Li, Zhenning Li, Cheng-Zhong Xu
Abstract
Safety-critical scenarios are central to evaluating autonomous driving systems, yet their rarity in naturalistic logs makes simulation-based stress testing indispensable. Most scenario generation methods treat surrounding agents as adversaries, but they either (i) induce failures without explicitly modeling vehicle-road physical limits, yielding visually extreme yet physically unsolvable crashes, or (ii) enforce physical feasibility or policy feasibility in isolation, which can over-focus on aggressive maneuvers or remain tied to a controller-dependent capability boundary. We propose ScenePilot, a feasibility-guided, boundary-driven framework that targets the boundary band: scenarios that are physically solvable in principle yet still cause the deployed autonomy stack to fail. We formulate generation as constrained multi-objective reinforcement learning, combining an RSS-derived physical-feasibility score with an online-learned AV-risk predictor , and introduce step-level feasibility-aware shielding to keep exploration near the feasibility boundary while avoiding infeasible artifacts. Experiments on SafeBench with multiple planners show that ScenePilot yields substantially higher collision rates (+6.2 percentage points) while preserving physical validity, and that adversarial fine-tuning on these boundary-band scenarios consistently reduces downstream crash rates. The code is available at https://github.com/QiyuRuan/ScenePilot.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext ec554166-31ea-48ac-a971-6ec762a4538dBuilds on11
- On Adversarial Robustness of Trajectory Prediction for Autonomous VehiclesQingzhao Zhang, Shengtuo Hu, Jiachen Sun, Qi Alfred Chen et al.CVPR 2022 · 132 citations
- Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic PriorDavis Rempe, Jonah Philion, Leonidas J. Guibas, Sanja Fidler et al.CVPR 2022 · 123 citations
- Safe Reinforcement Learning Using Advantage-Based InterventionNolan Wagener, Byron Boots, Ching-An ChengICML 2021 · 66 citations
- ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous VehiclesJiawei Zhang, Chejian Xu, Bo LiCVPR 2024 · 50 citations
- Towards Safe Reinforcement Learning with a Safety Editor PolicyHaonan Yu, Wei Xu, Haichao ZhangNeurIPS 2022 · 50 citations
Related papers
- Steerable Adversarial Scenario Generation through Test-Time Preference AlignmentTong Nie, Yuewen Mei, Yihong Tang, Junlin He et al.ICLR 2026 · 8 citations
- AdvSim: Generating Safety-Critical Scenarios for Self-Driving VehiclesJingkang Wang, Ava Pun, James Tu, Sivabalan Manivasagam et al.CVPR 2021
- DiffScene: Diffusion-Based Safety-Critical Scenario Generation for Autonomous VehiclesChejian Xu, Aleksandr Petiushko, Ding Zhao, Bo LiAAAI 2025 · 90 citations
- MG-Fuzz: Model-Guided Fuzzing for Unsafe Scenario Discovery in Autonomous Driving SystemsYulong Lyu, Ruiqi Hong, Jiawan Wang, Jun Sun et al.ISSTA 2026
- SimScale: Learning to Drive via Real-World Simulation at ScaleHaochen Tian, Tianyu Li, Haochen Liu, Jiazhi Yang et al.CVPR 2026 · 40 citations
