Achieving Near Instance-Optimality and Minimax-Optimality in Stochastic and Adversarial Linear Bandits Simultaneously
Chung-Wei Lee, Haipeng Luo, Chen-Yu Wei, Mengxiao Zhang, Xiaojin Zhang
Abstract
In this work, we develop linear bandit algorithms that automatically adapt to different environments. By plugging a novel loss estimator into the optimization problem that characterizes the instance-optimal strategy, our first algorithm not only achieves nearly instance-optimal regret in stochastic environments, but also works in corrupted environments with additional regret being the amount of corruption, while the state-of-the-art (Li et al., 2019) achieves neither instance-optimality nor the optimal dependence on the corruption amount. Moreover, by equipping this algorithm with an adversarial component and carefully-designed testings, our second algorithm additionally enjoys minimax-optimal regret in completely adversarial environments, which is the first of this kind to our knowledge. Finally, all our guarantees hold with high probability, while existing instance-optimal guarantees only hold in expectation.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 33a44cbd-935b-4d5c-a365-6db5552bbcecCited by top-tier papers32
- Nearly Optimal Algorithms for Linear Contextual Bandits with Adversarial CorruptionsJiafan He, Dongruo Zhou, Tong Zhang, Quanquan GuNeurIPS 2022 · 66 citations
- The best of both worlds: stochastic and adversarial episodic MDPs with unknown transitionTiancheng Jin, Longbo Huang, Haipeng LuoNeurIPS 2021 · 51 citations
- First-Order Regret in Reinforcement Learning with Linear Function Approximation: A Robust Estimation ApproachAndrew J. Wagenmaker, Yifang Chen, Max Simchowitz, Simon S. Du et al.ICML 2022 · 49 citations
- Corruption-Robust Algorithms with Uncertainty Weighting for Nonlinear Contextual Bandits and Markov Decision ProcessesChenlu Ye, Wei Xiong, Quanquan Gu, Tong ZhangICML 2023 · 40 citations
- Corruption-Robust Offline Reinforcement Learning with General Function ApproximationChenlu Ye, Rui Yang, Quanquan Gu, Tong ZhangNeurIPS 2023 · 37 citations
Builds on2
- Bias no more: high-probability data-dependent regret bounds for adversarial bandits and MDPsChung-Wei Lee, Haipeng Luo, Chen-Yu Wei, Mengxiao ZhangNeurIPS 2020 · 65 citations
- An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear BanditsAndrea Tirinzoni, Matteo Pirotta, Marcello Restelli, Alessandro LazaricNeurIPS 2020 · 37 citations
Related papers
- Corruption-Robust Linear Bandits: Minimax Optimality and Gap-Dependent MisspecificationHaolin Liu, Artin Tajdini, Andrew Wagenmaker, Chen-Yu WeiNeurIPS 2024 · 8 citations
- Robust Lipschitz Bandits to Adversarial CorruptionsYue Kang, Cho-Jui Hsieh, Thomas Chun Man LeeNeurIPS 2023 · 20 citations
- Adversarial Bandits with Corruptions: Regret Lower Bound and No-regret AlgorithmLin Yang, Mohammad Hassan Hajiesmaili, Mohammad Sadegh Talebi, John C. S. Lui et al.NeurIPS 2020 · 39 citations
- Hybrid Regret Bounds for Combinatorial Semi-Bandits and Adversarial Linear BanditsShinji ItoNeurIPS 2021 · 31 citations
- An Exploration-by-Optimization Approach to Best of Both Worlds in Linear BanditsShinji Ito, Kei TakemuraNeurIPS 2023 · 7 citations
