Gradient Shaping: Enhancing Backdoor Attack Against Reverse Engineering
Rui Zhu, Di Tang, Siyuan Tang, Zihao Wang, Guanhong Tao, Shiqing Ma, XiaoFeng Wang, Haixu Tang
Abstract
Most existing methods to detect backdoored machine learning (ML) models take one of the two approaches: trigger inversion (aka. reverse engineer) and weight analysis (aka. model diagnosis). In particular, the gradient-based trigger inversion is considered to be among the most effective backdoor detection techniques, as evidenced by the TrojAI competition, Trojan Detection Challenge and backdoorBench. However, little has been done to understand why this technique works so well and, more importantly, whether it raises the bar to the backdoor attack. In this paper, we report the first attempt to answer this question by analyzing the change rate of the backdoored model around its trigger-carrying inputs. Our study shows that existing attacks tend to inject the backdoor characterized by a low change rate around trigger-carrying inputs, which are easy to capture by gradient-based trigger inversion. In the meantime, we found that the low change rate is not necessary for a backdoor attack to succeed: we design a new attack enhancement called Gradient Shaping (GRASP), which follows the opposite direction of adversarial training to reduce the change rate of a backdoored model with regard to the trigger, without undermining its backdoor effect. Also, we provide a theoretic analysis to explain the effectiveness of this new technique and the fundamental weakness of gradient-based trigger inversion. Finally, we perform both theoretical and experimental analysis, showing that the GRASP enhancement does not reduce the effectiveness of the stealthy attacks against the backdoor detection methods based on weight analysis, as well as other backdoor mitigation methods without using detection.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Detecting AI Trojans Using Meta Neural Analysis cites Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
- Towards Backdoor Stealthiness in Model Parameter Space cites Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
- Towards Backdoor Stealthiness in Model Parameter Space cites ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation
- Towards Backdoor Stealthiness in Model Parameter Space cites Anti-Backdoor Learning: Training Clean Models on Poisoned Data
- BAN: Detecting Backdoors Activated by Adversarial Neuron Noise cites Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
- BAN: Detecting Backdoors Activated by Adversarial Neuron Noise cites ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation
- BAN: Detecting Backdoors Activated by Adversarial Neuron Noise cites Anti-Backdoor Learning: Training Clean Models on Poisoned Data
- ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation cites Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
- Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks cites Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
- Rounding-Guided Backdoor Injection in Deep Learning Model Quantization cites Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
- Rounding-Guided Backdoor Injection in Deep Learning Model Quantization cites Detecting AI Trojans Using Meta Neural Analysis
- Rounding-Guided Backdoor Injection in Deep Learning Model Quantization cites ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation
- Anti-Backdoor Learning: Training Clean Models on Poisoned Data cites Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
- Anti-Backdoor Learning: Training Clean Models on Poisoned Data cites Detecting AI Trojans Using Meta Neural Analysis
- Anti-Backdoor Learning: Training Clean Models on Poisoned Data cites Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
Cited by top-tier papers3
- BAN: Detecting Backdoors Activated by Adversarial Neuron NoiseXiaoyun Xu, Zhuoran Liu, Stefanos Koffas, Shujian Yu et al.NeurIPS 2024 · 13 citations
- Rounding-Guided Backdoor Injection in Deep Learning Model QuantizationXiangxiang Chen, Peixin Zhang, Jun Sun, Wenhai Wang et al.NDSS 2026 · 4 citations
- Towards Backdoor Stealthiness in Model Parameter SpaceXiaoyun Xu, Zhuoran Liu, Stefanos Koffas, Stjepan PicekCCS 2025
Builds on23
- Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural NetworksBolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li et al.S&P 2019 · 1,801 citations
- Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural NetworksYige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu et al.ICLR 2021 · 548 citations
- ABS: Scanning Neural Networks for Back-doors by Artificial Brain StimulationYingqi Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma et al.CCS 2019 · 531 citations
- Anti-Backdoor Learning: Training Clean Models on Poisoned DataYige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu et al.NeurIPS 2021 · 503 citations
- Detecting AI Trojans Using Meta Neural AnalysisXiaojun Xu, Qi Wang, Huichen Li, Nikita Borisov et al.S&P 2021 · 381 citations
Related papers
- OdScan: Backdoor Scanning for Object Detection ModelsSiyuan Cheng, Guangyu Shen, Guanhong Tao, Kaiyuan Zhang et al.S&P 2024 · 14 citations
- Complex Backdoor Detection by Symmetric Feature DifferencingYingqi Liu, Guangyu Shen, Guanhong Tao, Zhenting Wang et al.CVPR 2022 · 35 citations
- Single Image Backdoor Inversion via Robust Smoothed ClassifiersMingjie Sun, Zico KolterCVPR 2023
- Better Trigger Inversion Optimization in Backdoor ScanningGuanhong Tao, Guangyu Shen, Yingqi Liu, Shengwei An et al.CVPR 2022 · 47 citations
- Enhancing Fine-Tuning based Backdoor Defense with Sharpness-Aware MinimizationMingli Zhu, Shaokui Wei, Li Shen, Yanbo Fan et al.ICCV 2023 · 95 citations
