Trajectory-Aware Spiking DiTs Conversion via Membrane Potential Error-Feedback
Haoran Fang, Tianxing Man, Xingchen Li, Wanli Shi, Jinjie Fang, Bin Gu
摘要
Diffusion Transformers (DiTs) have achieved state-of-the-art generative performance, yet their iterative denoising process remains computationally expensive and energy-intensive. Spiking Neural Networks (SNNs) offer a promising neuromorphic alternative for energy efficiency; however, the non-differentiable nature of spiking neurons makes direct training difficult, positioning ANNto-SNN conversion as a more practical, trainingfree solution. In this paper, we identify a critical challenge unique to converting DiTs: standard fixed-scale spiking neurons fail to accommodate the highly dynamic activation ranges inherent across denoising steps. This mismatch leads to cumulative errors that significantly degrade generation fidelity. To resolve this, we propose a novel conversion framework featuring Multi-Threshold (MT) neurons and a Membrane Potential Error-Feedback (MPEF) mechanism. MT neurons expand the expressive capacity of discrete spikes by employing a multilevel firing strategy. Concurrently, MPEF exploits the temporal correlation between successive denoising steps to recycle residual membrane potential, effectively compensating for information loss and mitigating distribution shifts without retraining. Extensive experiments on ImageNet demonstrate that our framework achieves competitive generative quality with superior energy efficiency, establishing a new performance benchmark for spiking Diffusion Transformers. Our code is available at https://github.com/ JustVelkhana/SpikingDiT.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper17
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser 等CVPR 2022 · 被引用 13,123 次
- Scalable Diffusion Models with TransformersWilliam Peebles, Saining XieICCV 2023 · 被引用 5,568 次
- Scaling Rectified Flow Transformers for High-Resolution Image SynthesisPatrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari 等ICML 2024 · 被引用 3,620 次
- PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image SynthesisJunsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao 等ICLR 2024 · 被引用 831 次
相关 Paper
- Efficient ANN-SNN Conversion with Error Compensation LearningChang Liu, Jiangrong Shen, Xuming Ran, Mingkun Xu 等ICML 2025
- Towards Training-Free and Accurate ANN-to-SNN Conversion via Activation-Aware RedistributionHonglin Cao, Shuai Wang, Zijian Zhou, Ammar Belatreche 等AAAI 2026
- SpikingIR: A Novel Converted Spiking Neural Network for Efficient Image RestorationYang Ouyang, Zihan Cheng, Xiaotong Luo, Guoqi Li 等AAAI 2026
- Generalized Threshold Optimization with Harmony Multi-Threshold Neurons for Accurate ANN-to-SNN ConversionWenhan Zhang, Zihan Huang, Tong Bu, Tiejun Huang 等AAAI 2026
- Adaptive Calibration: A Unified Conversion Framework of Spiking Neural NetworksZiqing Wang, Yuetong Fang, Jiahang Cao, Hongwei Ren 等AAAI 2025 · 被引用 9 次
