QPRL : Learning Optimal Policies with Quasi-Potential Functions for Asymmetric Traversal
Jumman Hossain, Nirmalya Roy
Abstract
Reinforcement learning (RL) in real-world tasks such as robotic navigation often encounters environments with asymmetric traversal costs, where actions like climbing uphill versus moving downhill incur distinctly different penalties, or transitions may become irreversible. While recent quasimetric RL methods relax symmetry assumptions, they typically do not explicitly account for path-dependent costs or provide rigorous safety guarantees. We introduce Quasi-Potential Reinforcement Learning (QPRL), a novel framework that explicitly decomposes asymmetric traversal costs into a path-independent potential function (Φ) and a path-dependent residual (Ψ). This decomposition allows efficient learning and stable policy optimization via a Lyapunov-based safety mechanism. Theoretically, we prove that QPRL achieves convergence with improved sample complexity of Õ( √ T ), surpassing prior quasimetric RL bounds of Õ(T ). Empirically, our experiments demonstrate that QPRL attains state-of-theart performance across various navigation and control tasks, significantly reducing irreversible constraint violations by approximately 4× compared to baselines.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning cites Contrastive Learning as Goal-Conditioned Reinforcement Learning
- Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning cites On the Learning and Learnability of Quasimetrics
- Learning Anisotropic Value Geometry with Finsler Reinforcement Learning cites Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning
- On Reward-Free Reinforcement Learning with Linear Function Approximation cites Reward-Free Exploration for Reinforcement Learning
Cited by top-tier papers1
Builds on5
- Contrastive Learning as Goal-Conditioned Reinforcement LearningBenjamin Eysenbach, Tianjun Zhang, Sergey Levine, Ruslan SalakhutdinovNeurIPS 2022 · 331 citations
- Reward-Free Exploration for Reinforcement LearningChi Jin, Akshay Krishnamurthy, Max Simchowitz, Tiancheng YuICML 2020 · 226 citations
- On Reward-Free Reinforcement Learning with Linear Function ApproximationRuosong Wang, Simon S. Du, Lin F. Yang, Ruslan SalakhutdinovNeurIPS 2020 · 121 citations
- Optimal Goal-Reaching Reinforcement Learning via Quasimetric LearningTongzhou Wang, Antonio Torralba, Phillip Isola, Amy ZhangICML 2023 · 88 citations
- On the Learning and Learnability of QuasimetricsTongzhou Wang, Phillip IsolaICLR 2022 · 12 citations
Related papers
- Goal Reaching with Eikonal-Constrained Hierarchical Quasimetric Reinforcement LearningVittorio Giammarino, Ahmed Hussain QureshiICLR 2026 · 7 citations
- Conservative and Adaptive Penalty for Model-Based Safe Reinforcement LearningYecheng Jason Ma, Andrew Shen, Osbert Bastani, Dinesh JayaramanAAAI 2022 · 32 citations
- Proactive Constrained Policy Optimization with Preemptive PenaltyNing Yang, Pengyu Wang, Guoqing Liu, Haifeng Zhang et al.AAAI 2026 · 1 citation
- Tilted Quantile Gradient Updates for Quantile-Constrained Reinforcement LearningChenglin Li, Guangchun Ruan, Hua GengAAAI 2025
- Reward Penalties on Augmented States for Solving Richly Constrained RL EffectivelyHao Jiang, Tien Mai, Pradeep Varakantham, Huy HoangAAAI 2024 · 2 citations
