Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compression
Jung Yi, Wooseok Jang, Paul Cho, Jisu Nam, Heeji Yoon, Seungryong Kim
Abstract
Recent advances in autoregressive video diffusion have enabled real-time frame streaming, however, existing methods still suffer from visual error accumulation including visual fidelity and motion degradation over long-horizon. To address these challenges, we introduce Deep Forcing, a training-free extension of autoregressive video diffusion models that stabilizes long video generation through two complementary mechanisms. Deep Sink preserves approximately half of the sliding context window as persistent sink tokens and realigns their temporal RoPE phases to the current timeline, thereby maintaining global context during extended rollouts. Participative Compression performs importance-aware KV cache pruning, retaining only tokens that actively participate in recent attention while removing redundant or degraded history, effectively mitigating error accumulation under out-of-distribution lengths. Together, these components enable over 12× length extrapolation (e.g., 5s-trained → 60s+) without sacrificing inference speed, while improving visual fidelity and motion dynamics compared to prior methods. Our results demonstrate that Deep Forcing can achieve performance comparable to state-of-the-art training-based methods trained specifically for long video generation.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext f669f40d-2832-4202-9cfe-0c94dbfb0617Cited by top-tier papers4
- Mode Seeking meets Mean Seeking for Fast Long Video GenerationShengqu Cai, Weili Nie, Chao Liu, Julius Berner et al.ICML 2026 · 9 citations
- EMFormer: Efficient Multi-Scale Transformer for Accumulative Context Weather Forecastinghao chen, Tao Han, Jie ZHANG, Song Guo et al.ICML 2026
- Yume1.5: A Text-Controlled Interactive World Generation ModelXiaofeng Mao, Zhen Li, Chuanhao Li, Xiaojie Xu et al.CVPR 2026
- Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video GenerationHongzhou Zhu, Min Zhao, Guande He, Hang Su et al.ICML 2026
Builds on18
- The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations
- Efficient Streaming Language Models with Attention SinksGuangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han et al.ICLR 2024 · 1,714 citations
- H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language ModelsZhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen et al.NeurIPS 2023 · 1,003 citations
- Diffusion Forcing: Next-token Prediction Meets Full-Sequence DiffusionBoyuan Chen, Diego Marti Monso, Yilun Du, Max Simchowitz et al.NeurIPS 2024 · 751 citations
- InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and GenerationYi Wang, Yinan He, Yizhuo Li, Kunchang Li et al.ICLR 2024 · 467 citations
Related papers
- Self Forcing: Bridging the Train-Test Gap in Autoregressive Video DiffusionXun Huang, Zhengqi Li, Guande He, Mingyuan Zhou et al.NeurIPS 2025 · 628 citations
- Rolling Forcing: Autoregressive Long Video Diffusion in Real TimeKunhao Liu, Wenbo Hu, Jiale Xu, Ying Shan et al.ICLR 2026 · 215 citations
- LoL: Longer than Longer, Scaling Video Generation to HourJustin Cui, Jie Wu, Ming Li, Tao Yang et al.CVPR 2026 · 30 citations
- Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion ModelXinyin Ma, Julius Berner, Chao Liu, Arash Vahdat et al.ICML 2026
- Accelerating Autoregressive Video Diffusion via History-Guided Cache and Residual CorrectionKepan Nan, Wangbo Zhao, Penghao Zhou, Jun Li et al.CVPR 2026
