Optimization Method for Surrogate Function in Spiking Neural Networks Based on Membrane Potential Distribution
Qi Sun, Zhen Cao, Kaige Geng, Ziyi Zhang, Biao Hou
Abstract
Spiking Neural Networks (SNNs) offer promising energy efficiency and temporal sparsity for edge intelligence, but their training remains difficult due to gradient mismatch, membrane potential drift, and discretization errors. In this paper, we propose a membrane potential-guided surrogate optimization(MPO) framework that dynamically aligns the surrogate function with the membrane potential distribution to enhance the gradient propagation. Specifically, we introduce a KL-divergence-based regularization to stabilize membrane potential dynamics, and an adaptive width constraint to synchronize the surrogate gradient range with neural activity statistics. Additionally, we design a spike discretization error metric and a correction strategy to mitigate temporal discretization effects. Experiments on CIFAR-10, CIFAR-100, and ImageNet show our method achieves 94.76%, 74.20%, and 65.70% top-1 accuracy respectively, while improving gradient stability and energy efficiency. This work provides a principled optimization scheme for robust and scalable SNN training in practical neuromorphic systems.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext be885b9b-ce82-492e-8b0d-6830ef88e631Builds on14
- Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent BackpropagationNitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, Kaushik RoyICLR 2020 · 347 citations
- Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural NetworksYuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng et al.NeurIPS 2021 · 288 citations
- Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural NetworksTong Bu, Wei Fang, Jianhao Ding, Penglin Dai et al.ICLR 2022 · 272 citations
- Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural NetworksWenrui Zhang, Peng LiNeurIPS 2020 · 264 citations
- Temporal Effective Batch Normalization in Spiking Neural NetworksChaoteng Duan, Jianhao Ding, Shiyan Chen, Zhaofei Yu et al.NeurIPS 2022 · 141 citations
Related papers
- ASG: Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural NetworksYechan Kang, Yongjin Kweon, Mingyeong Seo, Sohee Park et al.ICML 2026
- DeepTAGE: Deep Temporal-Aligned Gradient Enhancement for Optimizing Spiking Neural NetworksWei Liu, Li Yang, Mingxuan Zhao, Shuxun Wang et al.ICLR 2025
- Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingShikuang Deng, Yuhang Li, Shanghang Zhang, Shi GuICLR 2022 · 361 citations
- Adaptive Smoothing Gradient Learning for Spiking Neural NetworksZiming Wang, Runhao Jiang, Shuang Lian, Rui Yan et al.ICML 2023 · 69 citations
- MPD-SGR: Robust Spiking Neural Networks with Membrane Potential Distribution-Driven Surrogate Gradient RegularizationRunhao Jiang, Chengzhi Jiang, Rui Yan, Huajin TangAAAI 2026 · 2 citations
