Model Reveals What to Cache: Profiling-Based Feature Reuse for Video Diffusion Models
Xuran Ma, Yexin Liu, Yaofu Liu, Xianfeng Wu, Mingzhe Zheng, Zihao Wang, Ser-Nam Lim, Harry Yang
摘要
Recent advances in diffusion models have demonstrated remarkable capabilities in video generation. However, the computational intensity remains a significant challenge for practical applications. While feature caching has been proposed to reduce the computational burden of diffusion models, existing methods typically overlook the heterogeneous significance of individual blocks, resulting in suboptimal reuse and degraded output quality. To this end, we address this gap by introducing ProfilingDiT, a novel adaptive caching strategy that explicitly disentangles foreground and background-focused blocks. Through a systematic analysis of attention distributions in diffusion models, we reveal a key observation: 1) Most layers exhibit a consistent preference for either foreground or background regions. 2) Predicted noise shows low inter-step similarity initially, which stabilizes as denoising progresses. This finding inspires us to formulate a selective caching strategy that preserves full computation for dynamic foreground elements while efficiently caching static background features. Our approach substantially reduces computational overhead while preserving visual fidelity. Extensive experiments demonstrate that our framework achieves significant acceleration (e.g., speedup for Wan2.1) while maintaining visual fidelity across comprehensive quality metrics, establishing a viable method for efficient video generation.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper5
- Follow-Your-Shape: Shape-Aware Image Editing via Trajectory-Guided Region ControlZeqian Long, Mingzhe Zheng, Kunyu Feng, Xinhua Zhang 等ICLR 2026 · 被引用 29 次
- ContextFlow: Training-Free Video Object Editing via Adaptive Context EnrichmentYiyang Chen, Xuanhua He, Xiujun Ma, Jack MaAAAI 2026 · 被引用 17 次
- BWCache: Accelerating Video Diffusion Transformers through Block-Wise CachingHanshuai Cui, Zhiqing Tang, Zhifei Xu, Zhi Yao 等ICLR 2026 · 被引用 11 次
- ERTACache: Error Rectification and Timesteps Adjustment for Efficient DiffusionXurui Peng, Chenqian Yan, Hong Liu, Rui Ma 等ICLR 2026 · 被引用 10 次
- RT-Lynx: Putting the GEMM Sparsity In a Right Way for Diffusion ModelsXing Cong, Hanlin Tang, Kan Liu, Lan Tao 等ICML 2026 · 被引用 1 次
它引用的顶会 Paper18
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- Denoising Diffusion Implicit ModelsJiaming Song, Chenlin Meng, Stefano ErmonICLR 2021 · 被引用 11,743 次
- Scalable Diffusion Models with TransformersWilliam Peebles, Saining XieICCV 2023 · 被引用 5,568 次
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessTri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra 等NeurIPS 2022 · 被引用 5,493 次
- DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 StepsCheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen 等NeurIPS 2022 · 被引用 2,653 次
相关 Paper
- Lightning Fast Caching-based Parallel Denoising Prediction for Accelerating Talking Head GenerationJianzhi Long, Wenhao Sun, Rong-Cheng Tu, Dacheng TaoAAAI 2026
- SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware CachingYasaman Haghighi, Alexandre AlahiCVPR 2026 · 被引用 5 次
- RAPID: Reusing Attention Sparsity with Inter-step Adaptation for Efficient Video DiffusionShangran Lin, Lu Lu, Jian Chen, Qiang LiuCVPR 2026
- DiCache: Let Diffusion Model Determine Its Own CacheJiazi Bu, Pengyang Ling, Yujie Zhou, Yibin Wang 等ICLR 2026 · 被引用 36 次
- Accelerating Diffusion-based Video Editing via Heterogeneous Caching: Beyond Full Computing at Sampled Denoising TimestepTianyi Liu, Ye Lu, Linfeng Zhang, Chen Cai 等CVPR 2026 · 被引用 2 次
