Deadline-Safe Reach-Avoid Control Synthesis for Cyber-Physical Systems with Reinforcement Learning
Mengyu Liu, Pengyuan Lu, Xin Chen, Oleg Sokolsky, Insup Lee, Fanxin Kong
Abstract
Meeting deadlines is a fundamental requirement of cyber-physical systems (CPS) in real-time applications to consolidate their reliability and effectiveness in executing timecritical tasks. Recent research have focused on applying reinforcement learning to synthesize controllers for real-time systems, particularly in terms of achieving fast reach-avoid. However, achieving fast behavior does not necessarily equate to meeting deadlines. Sometimes reinforcement learning agents are trying to maximize the total reward by exploiting the reward function, and thus performing unwanted behavior, known as reward hacking. Therefore, depending on the deadlines, it is possible to have fast controllers that miss the deadlines and slow controllers that meet the deadlines. To address the misalignment between fast and meeting deadlines, we investigate the relationship between as soon as possible (ASAP) and deadline-safe. Additionally, we formulate the problem into a new Markov decision process R-MDP including time to avoid non-Markovian rewards when considering deadlines. Furthermore, we have designed new reward functions that encourage the agent to meet the deadlines. Moreover, we evaluate our method on various benchmarks. The experiment results show the effectiveness of our method in ensuring deadline compliance without compromising safety.
Ask Lune
Know where this paper stands in minutes, not an afternoon.
Lune can read the top-tier papers that cite it and the ones closest to it, and tells you what it claims, what came after and where it falls short, each point cited to the paper it comes from. No skimming, no AI slop.
Related papers
- Deductive Synthesis of Reinforcement Learning Agents for Infinite Horizon TasksYuning Wang, He ZhuCAV 2025
- Catch Me If You Learn: Real-Time Attack Detection and Mitigation in Learning Enabled CPSIpsita Koley, Sunandan Adhikary, Soumyajit DeyRTSS 2021 · 8 citations
- Safe Reinforcement Learning by Imagining the Near FutureGarrett Thomas, Yuping Luo, Tengyu MaNeurIPS 2021 · 118 citations
- Learning Human Objectives by Evaluating Hypothetical BehaviorSiddharth Reddy, Anca D. Dragan, Sergey Levine, Shane Legg et al.ICML 2020 · 81 citations
- An Optimistic Algorithm for online CMDPS with Anytime Adversarial ConstraintsJiahui Zhu, Kihyun Yu, Dabeen Lee, Xin Liu et al.ICML 2025
