Pareto Optimal Risk-Agnostic Distributional Bandits with Heavy-Tail Rewards
Kyungjae Lee, Dohyeong Kim, Taehyun Cho, Chaeyeon Kim, Yunkyung Ko, Seungyub Han, Seokhun Ju, Dohyeok Lee, Sungbin Lim
Abstract
This paper addresses the problem of multi-risk measure agnostic multi-armed bandits in heavy-tailed reward settings. We propose a framework that leverages novel deviation inequalities for the 1-Wasserstein distance to construct confidence intervals for Lipschitz risk measures. The distributional LCB (DistLCB) algorithm is introduced, which achieves asymptotic optimality by deriving the first lower bounds for risk measure aware bandits with explicit sub-optimality gap dependencies. The DistLCB is further extended to multi-risk objectives, which enables Pareto-optimal solutions that consider multiple aspects of reward distributions. Additionally, we provide a regret analysis that includes both gap-dependent and gap-independent bounds for multi-risk settings. Experiments validate the effectiveness of the proposed methods in synthetic and real-world applications.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 06aa052a-e3a7-441b-9205-6b5aecc62d08Builds on8
- Concentration bounds for CVaR estimation: The cases of light-tailed and heavy-tailed distributionsPrashanth L. A., Krishna P. Jagannathan, Ravi Kumar KollaICML 2020 · 53 citations
- Optimal Thompson Sampling strategies for support-aware CVaR banditsDorian Baudry, Romain Gautron, Emilie Kaufmann, Odalric MaillardICML 2021 · 40 citations
- Optimal Algorithms for Stochastic Multi-Armed Bandits with Heavy Tailed RewardsKyungjae Lee, Hongjun Yang, Sungbin Lim, Songhwai OhNeurIPS 2020 · 34 citations
- Adaptive Best-of-Both-Worlds Algorithm for Heavy-Tailed Multi-Armed BanditsJiatai Huang, Yan Dai, Longbo HuangICML 2022 · 24 citations
- Quantile Bandits for Best Arms IdentificationMengyan Zhang, Cheng Soon OngICML 2021 · 13 citations
Related papers
- Optimal Best-Arm Identification Methods for Tail-Risk MeasuresShubhada Agrawal, Wouter M. Koolen, Sandeep JunejaNeurIPS 2021 · 34 citations
- Bayesian Regret Minimization in Offline BanditsMarek Petrik, Guy Tennenholtz, Mohammad GhavamzadehICML 2024
- Taming Heavy-Tailed Losses in Adversarial Bandits and the Best-of-Both-Worlds SettingDuo Cheng, Xingyu Zhou, Bo JiNeurIPS 2024 · 3 citations
- A Simple and Optimal Policy Design for Online Learning with Safety against Heavy-tailed RiskDavid Simchi-Levi, Zeyu Zheng, Feng ZhuNeurIPS 2022 · 7 citations
- Continuous Mean-Covariance BanditsYihan Du, Siwei Wang, Zhixuan Fang, Longbo HuangNeurIPS 2021 · 5 citations
