Mean Flow Distillation: Robust and Stable Distillation for Flow Matching Models
An Zhao, Shengyuan Zhang, Zhongjian Sun, Yixiang Zhou, Zejian Li, Ling Yang, Tianrun Chen, Lingyun Sun
摘要
Flow Matching models have demonstrated strong performance across a wide range of generative tasks. However, their reliance on ODE-based iterative sampling incurs substantial computational overhead in inference, which limits their applicability in real-time scenes. While distillation is a promising solution, existing approaches largely borrow from diffusion-based score matching, often failing to exploit the intrinsic geometric structure of flows and suffering from training instability, high variance, and degraded generation quality. In this paper, we propose Mean Flow Distillation (MFD), a novel distillation framework tailored for flow matching models. We theoretically demonstrate that MFD acts as a temporal low-pass filter, effectively suppressing the high-frequency optimization noise inherent in variational score distillation (VSD) while ensuring global trajectory consistency. We further prove the Mean Flow Matching Theorem, establishing that matching expected average velocities is sufficient for strict distribution alignment. Empirically, on challenging tasks of high-dimensional manifolds including 4D occupancy forecasting and text-toimage generation, MFD achieves state-of-the-art performance, enabling high-fidelity single-step generation. The implementation is available at https://github.com/happyw1nd/MFD .
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper30
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 被引用 35,902 次
- Scaling Rectified Flow Transformers for High-Resolution Image SynthesisPatrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari 等ICML 2024 · 被引用 3,620 次
- Consistency ModelsYang Song, Prafulla Dhariwal, Mark Chen, Ilya SutskeverICML 2023 · 被引用 1,720 次
- ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score DistillationZhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao 等NeurIPS 2023 · 被引用 1,498 次
- ImageReward: Learning and Evaluating Human Preferences for Text-to-Image GenerationJiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong 等NeurIPS 2023 · 被引用 1,310 次
相关 Paper
- SwiftVideo: A Unified Framework for Few-Step Video Generation Through Trajectory-Distribution AlignmentYanxiao Sun, Jiafu Wu, Yun Cao, Chengming Xu 等AAAI 2026 · 被引用 6 次
- VarFlow: Proper Scoring-Rule Diffusion Distillation via Energy MatchingHuiyang Shao, Xin Xia, Yuxi Ren, Xing Wang 等NeurIPS 2025 · 被引用 1 次
- FlowSteer: Guiding Few-Step Image Synthesis with Authentic TrajectoriesLei Ke, Hubery Yin, Gongye Liu, Zhengyao Lv 等CVPR 2026 · 被引用 2 次
- Transition Matching Distillation for Fast Video GenerationWeili Nie, Julius Berner, Nanye Ma, Chao Liu 等CVPR 2026 · 被引用 24 次
- Consistent Flow Distillation for Text-to-3D GenerationRunjie Yan, Yinbo Chen, Xiaolong WangICLR 2025
