C-Learning: Horizon-Aware Cumulative Accessibility Estimation
Panteha Naderian, Gabriel Loaiza-Ganem, Harry J. Braviner, Anthony L. Caterini, Jesse C. Cresswell, Tong Li, Animesh Garg
Abstract
Multi-goal reaching is an important problem in reinforcement learning needed to achieve algorithmic generalization. Despite recent advances in this field, current algorithms suffer from three major challenges: high sample complexity, learning only a single way of reaching the goals, and difficulties in solving complex motion planning tasks. In order to address these limitations, we introduce the concept of cumulative accessibility functions, which measure the reachability of a goal from a given state within a specified horizon. We show that these functions obey a recurrence relation, which enables learning from offline interactions. We also prove that optimal cumulative accessibility functions are monotonic in the planning horizon. Additionally, our method can trade off speed and reliability in goal-reaching by suggesting multiple paths to a single goal depending on the provided horizon. We evaluate our approach on a set of multi-goal discrete and continuous control tasks. We show that our method outperforms state-of-the-art goal-reaching algorithms in success rate, sample complexity, and path optimality. Our code is available at https://github.com/layer6ai-labs/CAE , and additional visualizations can be found at https://sites.google.com/view/ learning-cae/ .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on4
- Skew-Fit: State-Covering Self-Supervised Reinforcement LearningVitchyr Pong, Murtaza Dalal, Steven Lin, Ashvin Nair et al.ICML 2020 · 303 citations
- Learning to Reach Goals via Iterated Supervised LearningDibya Ghosh, Abhishek Gupta, Ashwin Reddy, Justin Fu et al.ICLR 2021 · 222 citations
- Cautious Adaptation For Reinforcement Learning in Safety-Critical SettingsJesse Zhang, Brian Cheung, Chelsea Finn, Sergey Levine et al.ICML 2020 · 65 citations
- Conservative Safety Critics for ExplorationHomanga Bharadhwaj, Aviral Kumar, Nicholas Rhinehart, Sergey Levine et al.ICLR 2021 · 32 citations
Related papers
- Computably Continuous Reinforcement-Learning Objectives Are PAC-LearnableCambridge Yang, Michael Littman, Michael CarbinAAAI 2023
- Solving Minimum-Cost Reach Avoid using Reinforcement LearningOswin So, Cheng Ge, Chuchu FanNeurIPS 2024 · 24 citations
- Horizon Reduction Makes RL ScalableSeohong Park, Kevin Frans, Deepinder Mann, Benjamin Eysenbach et al.NeurIPS 2025 · 60 citations
- CUDC: A Curiosity-Driven Unsupervised Data Collection Method with Adaptive Temporal Distances for Offline Reinforcement LearningChenyu Sun, Hangwei Qian, Chunyan MiaoAAAI 2024 · 1 citation
- Reachability Constrained Reinforcement LearningDongjie Yu, Haitong Ma, Sheng-bo Li, Jianyu ChenICML 2022 · 90 citations
