Lune

AAAI2020Top-tier venue

Absum: Simple Regularization Method for Reducing Structural Sensitivity of Convolutional Neural Networks

Sekitoshi Kanai, Yasutoshi Ida, Yasuhiro Fujiwara, Masanori Yamada, Shuichi Adachi

Abstract

We propose Absum, which is a regularization method for improving adversarial robustness of convolutional neural networks (CNNs). Although CNNs can accurately recognize images, recent studies have shown that the convolution operations in CNNs commonly have structural sensitivity to specific noise composed of Fourier basis functions. By exploiting this sensitivity, they proposed a simple black-box adversarial attack: Single Fourier attack. To reduce structural sensitivity, we can use regularization of convolution filter weights since the sensitivity of linear transform can be assessed by the norm of the weights. However, standard regularization methods can prevent minimization of the loss function because they impose a tight constraint for obtaining high robustness. To solve this problem, Absum imposes a loose constraint; it penalizes the absolute values of the summation of the parameters in the convolution layers. Absum can improve robustness against single Fourier attack while being as simple and efficient as standard regularization methods (e.g., weight decay and L1 regularization). Our experiments demonstrate that Absum improves robustness against single Fourier attack more than standard regularization methods. Furthermore, we reveal that robust CNNs with Absum are more robust against transferred attacks due to decreasing the common sensitivity and against high-frequency noise than standard regularization methods. We also reveal that Absum can improve robustness against gradient-based attacks (projected gradient descent) when used with adversarial training.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Can DBNNs Robust to Environmental Noise for Resource-constrained Scenarios?Wendong Zheng, Junyang Chen, Husheng Guo, Wenjian WangICML 2025

Builds on2

  1. Distillation as a Defense to Adversarial Perturbations Against Deep Neural NetworksNicolas Papernot, Patrick D. McDaniel, Xi Wu, Somesh Jha et al.S&P 2016 · 3,275 citations
  2. High-Frequency Component Helps Explain the Generalization of Convolutional Neural NetworksHaohan Wang, Xindi Wu, Zeyi Huang, Eric P. XingCVPR 2020

Related papers

  1. Convolutional Normalization: Improving Deep Convolutional Network Robustness and TrainingSheng Liu, Xiao Li, Yuexiang Zhai, Chong You et al.NeurIPS 2021 · 30 citations
  2. Large Norms of CNN Layers Do Not Hurt Adversarial RobustnessYouwei Liang, Dong HuangAAAI 2021 · 13 citations
  3. Improving Transferability of Adversarial Patches on Face Recognition With Generative ModelsZihao Xiao, Xianfeng Gao, Chilin Fu, Yinpeng Dong et al.CVPR 2021
  4. LEA2: A Lightweight Ensemble Adversarial Attack via Non-overlapping Vulnerable Frequency RegionsYaguan Qian, Shuke He, Chenyu Zhao, Jiaqiang Sha et al.ICCV 2023 · 26 citations
  5. Not All Low-Pass Filters are Robust in Graph Convolutional NetworksHeng Chang, Yu Rong, Tingyang Xu, Yatao Bian et al.NeurIPS 2021 · 65 citations
Dusk over the sea between two cliffs drawn in fine vertical lines