Lune

ICML2023Top-tier venue

On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm

Julien Aubert, Luc Lehéricy, Patricia Reynaud-Bouret

Abstract

When fitting the learning data of an individual to algorithm-like learning models, the observations are so dependent and non-stationary that one may wonder what the classical Maximum Likelihood Estimator (MLE) could do, even if it is the usual tool applied to experimental cognition. Our objective in this work is to show that the estimation of the learning rate cannot be efficient if the learning rate is constant in the classical Exp3 (Exponential weights for Exploration and Exploitation) algorithm. Secondly, we show that if the learning rate decreases polynomially with the sample size, then the prediction error and in some cases the estimation error of the MLE satisfy bounds in probability that decrease at a polynomial rate.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Related papers

  1. Instance-Dependent Continuous-Time Reinforcement Learning via Maximum Likelihood EstimationRunze Zhao, Yue Yu, Ruhan Wang, Chunfeng Huang et al.ICML 2026 · 1 citation
  2. Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise ComparisonsBanghua Zhu, Michael I. Jordan, Jiantao JiaoICML 2023 · 273 citations
  3. Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy LearningAurélien Bibaut, Nathan Kallus, Maria Dimakopoulou, Antoine Chambaz et al.NeurIPS 2021 · 18 citations
  4. Modeling Human Exploration Through Resource-Rational Reinforcement LearningMarcel Binz, Eric SchulzNeurIPS 2022 · 23 citations
  5. Inferring learning rules from animal decision-makingZoe Ashwood, Nicholas A. Roy, Ji Hyun Bak, Jonathan W. PillowNeurIPS 2020 · 33 citations
Dusk over the sea between two cliffs drawn in fine vertical lines