R-learning in actor-critic model offers a biologically relevant mechanism for sequential decision-making
Sergey Shuvaev, Sarah Starosta, Duda Kvitsiani, Ádám Kepecs, Alexei A. Koulakov
Abstract
When should you continue with your ongoing plans and when should you instead decide to pursue better opportunities? We show in theory and experiment that such stay-or-leave decisions are consistent with deep R-learning both behaviorally and neuronally. Our results suggest that real-world agents leave depleting resources when their reward rate falls below its exponential average, which, we argue, is a Bayes optimal rule in dynamic natural environments. Our work links reinforcement learning, the marginal value theorem and Bayesian inference approaches to offer a learning algorithm and a decision rule for making sequential stay-or-leave choices.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers1
Related papers
- Dynamic allocation of limited memory resources in reinforcement learningNisheet Patel, Luigi Acerbi, Alexandre PougetNeurIPS 2020 · 6 citations
- Reinforcement Learning with Non-Exponential DiscountingMatthias Schultheis, Constantin A. Rothkopf, Heinz KoepplNeurIPS 2022 · 19 citations
- Flexible inference for animal learning rules using neural networksYuhan Helena Liu, Victor Geadah, Jonathan W. PillowNeurIPS 2025
- Inverse Rational Control with Partially Observable Continuous Nonlinear DynamicsMinhae Kwon, Saurabh Daptardar, Paul R. Schrater, Xaq PitkowNeurIPS 2020 · 46 citations
- Learning to Stop: Deep Learning for Mean Field Optimal StoppingLorenzo Magnino, Yuchen Zhu, Mathieu LaurièreICML 2025
