Can DBNNs Robust to Environmental Noise for Resource-constrained Scenarios?
Wendong Zheng, Junyang Chen, Husheng Guo, Wenjian Wang
Abstract
Recently, the potential of lightweight models for resource-constrained scenarios has garnered significant attention, particularly in safety-critical tasks such as bio-electrical signal classification and B-ultrasound-assisted diagnostic. These tasks are frequently affected by environmental noise due to patient movement artifacts and inherent device noise, which pose significant challenges for lightweight models (e.g., deep binary neural networks (DBNNs)) to perform robust inference. A pertinent question arises: can a well-trained DBNN effectively resist environmental noise during inference? In this study, we find that the DBNN's robustness vulnerability comes from the binary weights and scaling factors Drawing upon theoretical insights, we propose L1-infinite norm constraints for binary weights and scaling factors, which yield a tighter upper bound compared to existing state-of-the-art (SOTA) methods. Finally, visualization studies show that our approach introduces minimal noise perturbations at the periphery of the feature maps. Our approach outperforms the SOTA method, as validated by several experiments conducted on the bio-electrical and image classification datasets. We hope our findings can raise awareness among researchers about the environmental noise robustness of DBNNs.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on20
- Towards Stable and Efficient Training of Verifiably Robust Neural NetworksHuan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal et al.ICLR 2020 · 384 citations
- Adversarial Training and Provable Defenses: Bridging the GapMislav Balunovic, Martin T. VechevICLR 2020 · 186 citations
- Quantitative Verification of Neural Networks and Its Security ApplicationsTeodora Baluta, Shiqi Shen, Shweta Shinde, Kuldeep S. Meel et al.CCS 2019 · 115 citations
- Robust Quantization: One Model to Rule Them AllMoran Shkolnik, Brian Chmiel, Ron Banner, Gil Shomron et al.NeurIPS 2020 · 103 citations
- How Do Adam and Training Strategies Help BNNs OptimizationZechun Liu, Zhiqiang Shen, Shichao Li, Koen Helwegen et al.ICML 2021 · 100 citations
Related papers
- Towards Certificated Model Robustness Against Weight PerturbationsTsui-Wei Weng, Pu Zhao, Sijia Liu, Pin-Yu Chen et al.AAAI 2020 · 33 citations
- Binarized Low-Light Raw Video EnhancementGengchen Zhang, Yulun Zhang, Xin Yuan, Ying FuCVPR 2024 · 10 citations
- Bayesian Optimized 1-Bit CNNsJiaxin Gu, Junhe Zhao, Xiaolong Jiang, Baochang Zhang et al.ICCV 2019 · 57 citations
- BDD4BNN: A BDD-Based Quantitative Analysis Framework for Binarized Neural NetworksYedi Zhang, Zhe Zhao, Guangke Chen, Fu Song et al.CAV 2021 · 26 citations
- Specifying Weight Priors in Bayesian Deep Neural Networks with Empirical BayesRanganath Krishnan, Mahesh Subedar, Omesh TickooAAAI 2020 · 65 citations
