Diversity-aware Weight Perturbation Promotes Robust Adaptation
Zibo Chen, Ruxin Li, Zilu Wang
摘要
Compute-In-Memory (CIM) accelerators are promising for energy-efficient edge inference, yet they faces fundamental challenges when deploying Deep Neural Networks (DNNs), as hardware-induced weight perturbations from intrinsic noise and device drift degrade accuracy and impede reliable inference. To tackle this challenge, we propose Diversity-aware Weight Perturbation (DWP), an immune-system-inspired training method that emulates affinity-based selection by exploiting sample-level prediction disagreement under diverse noise realizations to guide adaptive sample weighting, building robustness to weight perturbation. Experiments show that DWP-trained models consistently yield superior robustness, achieving over 15% accuracy improvements compared to standard-trained models under severe weight perturbations (mismatch level up to 70%) and maintaining inference accuracy at 90% over a simulated one-year CIM operation with only 2%–4% variation in accuracy. Moreover, under matched model and inference configurations, deployment on low-precision CIM hardware reduces inference energy by 38% compared to a GPU baseline. These results demonstrate that DWP enables robust and energy-efficient neural network deployment on resource-constrained edge devices with inherent hardware uncertainties.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper8
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn 等ICLR 2021 · 被引用 21,477 次
- Adversarial Weight Perturbation Helps Robust GeneralizationDongxian Wu, Shu-Tao Xia, Yisen WangNeurIPS 2020 · 被引用 917 次
- Improving Out-of-Distribution Robustness via Selective AugmentationHuaxiu Yao, Yu Wang, Sai Li, Linjun Zhang 等ICML 2022 · 被引用 275 次
- One-bit Flip is All You Need: When Bit-flip Attack Meets Model TrainingJianshuo Dong, Han Qiu, Yiming Li, Tianwei Zhang 等ICCV 2023 · 被引用 33 次
- Connecting Certified and Adversarial TrainingYuhao Mao, Mark Niklas Müller, Marc Fischer, Martin T. VechevNeurIPS 2023 · 被引用 14 次
相关 Paper
- Leveraging Noise and Aggressive Quantization of In-Memory Computing for Robust DNN Hardware Against Adversarial Input and Weight AttacksSai Kiran Cherupally, Adnan Siraj Rakin, Shihui Yin, Mingoo Seok 等DAC 2021 · 被引用 10 次
- Network Insensitivity to Parameter Noise via Parameter Attack During TrainingJulian Büchel, Fynn Firouz Faber, Dylan Richard MuirICLR 2022 · 被引用 6 次
- TFix: Exploiting the Natural Redundancy of Ternary Neural Networks for Fault Tolerant In-Memory Vector Matrix MultiplicationAkul Malhotra, Chunguang Wang, Sumeet Kumar GuptaDAC 2023 · 被引用 4 次
- Improving robustness to corruptions with multiplicative weight perturbationsTrung Q. Trinh, Markus Heinonen, Luigi Acerbi, Samuel KaskiNeurIPS 2024 · 被引用 8 次
- In-Memory Neural Network Accelerator based on eDRAM Cell with Enhanced Retention TimeInhwan Lee, Eunhwan Kim, Nameun Kang, Hyunmyung Oh 等DAC 2023 · 被引用 6 次
