Achieving Near Instance-Optimality and Minimax-Optimality in Stochastic and Adversarial Linear Bandits Simultaneously
Chung-Wei Lee, Haipeng Luo, Chen-Yu Wei, Mengxiao Zhang, Xiaojin Zhang
摘要
In this work, we develop linear bandit algorithms that automatically adapt to different environments. By plugging a novel loss estimator into the optimization problem that characterizes the instance-optimal strategy, our first algorithm not only achieves nearly instance-optimal regret in stochastic environments, but also works in corrupted environments with additional regret being the amount of corruption, while the state-of-the-art (Li et al., 2019) achieves neither instance-optimality nor the optimal dependence on the corruption amount. Moreover, by equipping this algorithm with an adversarial component and carefully-designed testings, our second algorithm additionally enjoys minimax-optimal regret in completely adversarial environments, which is the first of this kind to our knowledge. Finally, all our guarantees hold with high probability, while existing instance-optimal guarantees only hold in expectation.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper32
- Nearly Optimal Algorithms for Linear Contextual Bandits with Adversarial CorruptionsJiafan He, Dongruo Zhou, Tong Zhang, Quanquan GuNeurIPS 2022 · 被引用 66 次
- The best of both worlds: stochastic and adversarial episodic MDPs with unknown transitionTiancheng Jin, Longbo Huang, Haipeng LuoNeurIPS 2021 · 被引用 51 次
- First-Order Regret in Reinforcement Learning with Linear Function Approximation: A Robust Estimation ApproachAndrew J. Wagenmaker, Yifang Chen, Max Simchowitz, Simon S. Du 等ICML 2022 · 被引用 49 次
- Corruption-Robust Algorithms with Uncertainty Weighting for Nonlinear Contextual Bandits and Markov Decision ProcessesChenlu Ye, Wei Xiong, Quanquan Gu, Tong ZhangICML 2023 · 被引用 40 次
- Corruption-Robust Offline Reinforcement Learning with General Function ApproximationChenlu Ye, Rui Yang, Quanquan Gu, Tong ZhangNeurIPS 2023 · 被引用 37 次
它引用的顶会 Paper2
- Bias no more: high-probability data-dependent regret bounds for adversarial bandits and MDPsChung-Wei Lee, Haipeng Luo, Chen-Yu Wei, Mengxiao ZhangNeurIPS 2020 · 被引用 65 次
- An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear BanditsAndrea Tirinzoni, Matteo Pirotta, Marcello Restelli, Alessandro LazaricNeurIPS 2020 · 被引用 37 次
相关 Paper
- Corruption-Robust Linear Bandits: Minimax Optimality and Gap-Dependent MisspecificationHaolin Liu, Artin Tajdini, Andrew Wagenmaker, Chen-Yu WeiNeurIPS 2024 · 被引用 8 次
- Robust Lipschitz Bandits to Adversarial CorruptionsYue Kang, Cho-Jui Hsieh, Thomas Chun Man LeeNeurIPS 2023 · 被引用 20 次
- Adversarial Bandits with Corruptions: Regret Lower Bound and No-regret AlgorithmLin Yang, Mohammad Hassan Hajiesmaili, Mohammad Sadegh Talebi, John C. S. Lui 等NeurIPS 2020 · 被引用 39 次
- Hybrid Regret Bounds for Combinatorial Semi-Bandits and Adversarial Linear BanditsShinji ItoNeurIPS 2021 · 被引用 31 次
- An Exploration-by-Optimization Approach to Best of Both Worlds in Linear BanditsShinji Ito, Kei TakemuraNeurIPS 2023 · 被引用 7 次
