Lune

NeurIPS2024Top-tier venue

Robust Neural Contextual Bandit against Adversarial Corruptions

Yunzhe Qi, Yikun Ban, Arindam Banerjee, Jingrui He

Abstract

Contextual bandit algorithms aim to identify the optimal arm with the highest reward among a set of candidates, based on the accessible contextual information. Among these algorithms, neural contextual bandit methods have shown generally superior performances against linear and kernel ones, due to the representation power of neural networks. However, similar to other neural network applications, neural bandit algorithms can be vulnerable to adversarial attacks or corruptions on the received labels (i.e., arm rewards), which can lead to unexpected performance degradation without proper treatments. As a result, it is necessary to improve the robustness of neural bandit models against potential reward corruptions. In this work, we propose a novel neural contextual bandit algorithm named R-NeuralUCB, which utilizes a novel context-aware Gradient Descent (GD) training strategy to improve the robustness against adversarial reward corruptions. Under over-parameterized neural network settings, we provide regret analysis for R-NeuralUCB to quantify reward corruption impacts, without the commonly adopted arm separateness assumption in existing neural bandit works. We also conduct experiments against baselines on real data sets under different scenarios, in order to demonstrate the effectiveness of our proposed R-NeuralUCB.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Faster, Smaller, and Smarter: Task-Aware Expert Merging for Online MoE Inference cites Neural Contextual Bandits with UCB-based Exploration
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Faster, Smaller, and Smarter: Task-Aware Expert Merging for Online MoE InferenceZiyi Han, Xutong Liu, Ruiting Zhou, Xiangxiang Dai et al.INFOCOM 2026 · 3 citations

Builds on40

  1. Label-Only Membership Inference AttacksChristopher A. Choquette-Choo, Florian Tramèr, Nicholas Carlini, Nicolas PapernotICML 2021 · 628 citations
  2. Neural Contextual Bandits with UCB-based ExplorationDongruo Zhou, Lihong Li, Quanquan GuICML 2020 · 329 citations
  3. Certified Robustness to Label-Flipping Attacks via Randomized SmoothingElan Rosenfeld, Ezra Winston, Pradeep Ravikumar, J. Zico KolterICML 2020 · 182 citations
  4. Learning with Good Feature Representations in Bandits and in RL with a Generative ModelTor Lattimore, Csaba Szepesvári, Gellért WeiszICML 2020 · 181 citations
  5. An Embarrassingly Simple Approach for Trojan Attack in Deep Neural NetworksRuixiang Tang, Mengnan Du, Ninghao Liu, Fan Yang et al.KDD 2020 · 164 citations

Related papers

  1. Learning Neural Contextual Bandits through Perturbed RewardsYiling Jia, Weitong Zhang, Dongruo Zhou, Quanquan Gu et al.ICLR 2022 · 20 citations
  2. Neural Bandit with Arm Group GraphYunzhe Qi, Yikun Ban, Jingrui HeKDD 2022 · 4 citations
  3. Meta Clustering of Neural BanditsYikun Ban, Yunzhe Qi, Tianxin Wei, Lihui Liu et al.KDD 2024 · 6 citations
  4. Robust Lipschitz Bandits to Adversarial CorruptionsYue Kang, Cho-Jui Hsieh, Thomas Chun Man LeeNeurIPS 2023 · 20 citations
  5. EE-Net: Exploitation-Exploration Neural Networks in Contextual BanditsYikun Ban, Yuchen Yan, Arindam Banerjee, Jingrui HeICLR 2022 · 62 citations
Dusk over the sea between two cliffs drawn in fine vertical lines