High-Fidelity ANN-to-SNN Conversion via Closed-Loop CKA Distillation
Bozhou Li, Chubo Liu, Yan Ding, Yufeng Zhang, Zhuo Tang, Kenli Li
摘要
ANN-to-SNN conversion offers energy-efficient inference but faces a fidelity-latency trade-off due to open-loop error accumulation. While conversion-aware training mitigates this, it sacrifices the generality of using off-the-shelf ANNs. We propose a closed-loop fine-tuning framework that calibrates these errors without altering the source model. Our approach employs a Dual Alignment Mechanism, utilizing global Kullback-Leibler divergence for output distillation and introducing an adaptive local Centered Kernel Alignment constraint, weighted by initial conversion loss, for feature alignment. We uncover a critical time-dependent dynamic: local constraints are essential for stabilizing representations in low-latency regimes (e.g., ) where global gradients are unstable, whereas global alignment drives fidelity at higher time steps. Experiments on CIFAR-10 demonstrate that our method achieves over 99% of source ANN accuracy at (e.g., ResNet-18: 96.38% vs. 96.39%). Furthermore, this fine-tuning acts as a regularizer, yielding SNNs with input noise robustness that matches or exceeds the source ANN.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper15
- Deep Residual Learning in Spiking Neural NetworksWei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang 等NeurIPS 2021 · 被引用 857 次
- Going Deeper With Directly-Trained Larger Spiking Neural NetworksHanle Zheng, Yujie Wu, Lei Deng, Yifan Hu 等AAAI 2021 · 被引用 694 次
- Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingShikuang Deng, Yuhang Li, Shanghang Zhang, Shi GuICLR 2022 · 被引用 361 次
- Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent BackpropagationNitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, Kaushik RoyICLR 2020 · 被引用 347 次
- Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural NetworksYuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng 等NeurIPS 2021 · 被引用 288 次
相关 Paper
- Efficient ANN-SNN Conversion with Error Compensation LearningChang Liu, Jiangrong Shen, Xuming Ran, Mingkun Xu 等ICML 2025
- Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural NetworksShikuang Deng, Shi GuICLR 2021 · 被引用 100 次
- Faster and Stronger: When ANN-SNN Conversion Meets Parallel Spiking CalculationZecheng Hao, Qichao Ma, Kang Chen, Yi Zhang 等ICML 2025
- A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks CalibrationYuhang Li, Shikuang Deng, Xin Dong, Ruihao Gong 等ICML 2021 · 被引用 239 次
- Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise ReplacementShu Yang, Chengting Yu, Lei Liu, Hanzhi Ma 等CVPR 2025
