Optimization Method for Surrogate Function in Spiking Neural Networks Based on Membrane Potential Distribution
Qi Sun, Zhen Cao, Kaige Geng, Ziyi Zhang, Biao Hou
摘要
Spiking Neural Networks (SNNs) offer promising energy efficiency and temporal sparsity for edge intelligence, but their training remains difficult due to gradient mismatch, membrane potential drift, and discretization errors. In this paper, we propose a membrane potential-guided surrogate optimization(MPO) framework that dynamically aligns the surrogate function with the membrane potential distribution to enhance the gradient propagation. Specifically, we introduce a KL-divergence-based regularization to stabilize membrane potential dynamics, and an adaptive width constraint to synchronize the surrogate gradient range with neural activity statistics. Additionally, we design a spike discretization error metric and a correction strategy to mitigate temporal discretization effects. Experiments on CIFAR-10, CIFAR-100, and ImageNet show our method achieves 94.76%, 74.20%, and 65.70% top-1 accuracy respectively, while improving gradient stability and energy efficiency. This work provides a principled optimization scheme for robust and scalable SNN training in practical neuromorphic systems.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper14
- Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent BackpropagationNitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, Kaushik RoyICLR 2020 · 被引用 347 次
- Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural NetworksYuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng 等NeurIPS 2021 · 被引用 288 次
- Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural NetworksTong Bu, Wei Fang, Jianhao Ding, Penglin Dai 等ICLR 2022 · 被引用 272 次
- Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural NetworksWenrui Zhang, Peng LiNeurIPS 2020 · 被引用 264 次
- Temporal Effective Batch Normalization in Spiking Neural NetworksChaoteng Duan, Jianhao Ding, Shiyan Chen, Zhaofei Yu 等NeurIPS 2022 · 被引用 141 次
相关 Paper
- ASG: Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural NetworksYechan Kang, Yongjin Kweon, Mingyeong Seo, Sohee Park 等ICML 2026
- DeepTAGE: Deep Temporal-Aligned Gradient Enhancement for Optimizing Spiking Neural NetworksWei Liu, Li Yang, Mingxuan Zhao, Shuxun Wang 等ICLR 2025
- Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingShikuang Deng, Yuhang Li, Shanghang Zhang, Shi GuICLR 2022 · 被引用 361 次
- Adaptive Smoothing Gradient Learning for Spiking Neural NetworksZiming Wang, Runhao Jiang, Shuang Lian, Rui Yan 等ICML 2023 · 被引用 69 次
- MPD-SGR: Robust Spiking Neural Networks with Membrane Potential Distribution-Driven Surrogate Gradient RegularizationRunhao Jiang, Chengzhi Jiang, Rui Yan, Huajin TangAAAI 2026 · 被引用 2 次
