Lune

CVPR2023Top-tier venue

Efficient Loss Function by Minimizing the Detrimental Effect of Floating-Point Errors on Gradient-Based Attacks

Yunrui Yu, Cheng-Zhong Xu

Abstract

Attackers can deceive neural networks by adding human imperceptive perturbations to their input data; this reveals the vulnerability and weak robustness of current deep-learning networks. Many attack techniques have been proposed to evaluate the model's robustness. Gradientbased attacks suffer from severely overestimating the robustness. This paper identifies that the relative error in calculated gradients caused by floating-point errors, including floating-point underflow and rounding errors, is a fundamental reason why gradient-based attacks fail to accurately assess the model's robustness. Although it is hard to eliminate the relative error in the gradients, we can control its effect on the gradient-based attacks. Correspondingly, we propose an efficient loss function by minimizing the detrimental impact of the floating-point errors on the attacks. Experimental results show that it is more efficient and reliable than other loss functions when examined across a wide range of defence mechanisms.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual Attacks cites Towards Evaluating the Robustness of Neural Networks
  • Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual Attacks cites Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
  • Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks cites Towards Evaluating the Robustness of Neural Networks
  • Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks cites Fast is better than free: Revisiting adversarial training
  • Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks cites Overfitting in adversarially robust deep learning
  • Fast is better than free: Revisiting adversarial training cites Towards Evaluating the Robustness of Neural Networks
  • Towards Understanding and Improving Adversarial Robustness of Vision Transformers cites Towards Evaluating the Robustness of Neural Networks
  • Towards Understanding and Improving Adversarial Robustness of Vision Transformers cites Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
  • Towards Understanding and Improving Adversarial Robustness of Vision Transformers cites On Adaptive Attacks to Adversarial Example Defenses
  • On Adaptive Attacks to Adversarial Example Defenses cites Towards Evaluating the Robustness of Neural Networks
  • On Adaptive Attacks to Adversarial Example Defenses cites Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
  • Overfitting in adversarially robust deep learning cites Towards Evaluating the Robustness of Neural Networks
  • Overfitting in adversarially robust deep learning cites Fast is better than free: Revisiting adversarial training
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual AttacksYong Xie, Weijie Zheng, Hanxun Huang, Guangnan Ye et al.CVPR 2025
  2. Towards Understanding and Improving Adversarial Robustness of Vision TransformersSamyak Jain, Tanima DuttaCVPR 2024

Builds on22

  1. Towards Evaluating the Robustness of Neural NetworksNicholas Carlini, David A. WagnerS&P 2017 · 9,786 citations
  2. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacksFrancesco Croce, Matthias HeinICML 2020 · 2,337 citations
  3. Fast is better than free: Revisiting adversarial trainingEric Wong, Leslie Rice, J. Zico KolterICLR 2020 · 1,352 citations
  4. On Adaptive Attacks to Adversarial Example DefensesFlorian Tramèr, Nicholas Carlini, Wieland Brendel, Aleksander MadryNeurIPS 2020 · 1,026 citations
  5. Overfitting in adversarially robust deep learningLeslie Rice, Eric Wong, J. Zico KolterICML 2020 · 935 citations

Related papers

  1. Fooling a Complete Neural Network VerifierDániel Zombori, Balázs Bánhelyi, Tibor Csendes, István Megyeri et al.ICLR 2021 · 21 citations
  2. Floating-Point Networks with Automatic Differentiation Can Represent Almost All Floating-Point Functions and Their GradientsSejun Park, Yeachan Park, Geonho HwangICML 2026
  3. Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter CorruptionXu Sun, Zhiyuan Zhang, Xuancheng Ren, Ruixuan Luo et al.AAAI 2021 · 45 citations
  4. Improved Gradient-Based Adversarial Attacks for Quantized NetworksKartik Gupta, Thalaiyasingam AjanthanAAAI 2022 · 22 citations
  5. SuperDeepFool: a new fast and accurate minimal adversarial attackAlireza Abdollahpour, Mahed Abroshan, Seyed-Mohsen Moosavi-DezfooliNeurIPS 2024 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines