Improving Consistency Models with Generator-Augmented Flows
Thibaut Issenhuth, Sangchul Lee, Ludovic Dos Santos, Jean-Yves Franceschi, Chansoo Kim, Alain Rakotomamonjy
Abstract
Consistency models imitate the multi-step sampling of score-based diffusion in a single forward pass of a neural network. They can be learned in two ways: consistency distillation and consistency training. The former relies on the true velocity field of the corresponding differential equation, approximated by a pre-trained neural network. In contrast, the latter uses a single-sample Monte Carlo estimate of this velocity field. The related estimation error induces a discrepancy between consistency distillation and training that, we show, still holds in the continuous-time limit. To alleviate this issue, we propose a novel flow that transports noisy data towards their corresponding outputs derived from a consistency model. We prove that this flow reduces the previously identified discrepancy and the noise-data transport cost. Consequently, our method not only accelerates consistency training convergence but also enhances its overall performance. The code is available at: github.com/thibautissenhuth/consistency GC.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Elucidating the Design Space of Diffusion-Based Generative Models cites Denoising Diffusion Probabilistic Models
- Elucidating the Design Space of Diffusion-Based Generative Models cites Score-Based Generative Modeling through Stochastic Differential Equations
- VCT: Training Consistency Models with Variational Noise Coupling cites Elucidating the Design Space of Diffusion-Based Generative Models
- VCT: Training Consistency Models with Variational Noise Coupling cites Consistency Models
- VCT: Training Consistency Models with Variational Noise Coupling cites Denoising Diffusion Probabilistic Models
- VCT: Training Consistency Models with Variational Noise Coupling cites Score-Based Generative Modeling through Stochastic Differential Equations
- Consistency Models cites Elucidating the Design Space of Diffusion-Based Generative Models
- Consistency Models cites Denoising Diffusion Probabilistic Models
- Consistency Models cites Score-Based Generative Modeling through Stochastic Differential Equations
- Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling cites Denoising Diffusion Probabilistic Models
- Score-Based Generative Modeling through Stochastic Differential Equations cites Denoising Diffusion Probabilistic Models
Cited by top-tier papers1
Builds on19
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- Elucidating the Design Space of Diffusion-Based Generative ModelsTero Karras, Miika Aittala, Timo Aila, Samuli LaineNeurIPS 2022 · 3,959 citations
- Consistency ModelsYang Song, Prafulla Dhariwal, Mark Chen, Ilya SutskeverICML 2023 · 1,720 citations
- Score-Based Generative Modeling through Stochastic Differential EquationsYang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar et al.ICLR 2021 · 1,270 citations
- Diffusion Schrödinger Bridge with Applications to Score-Based Generative ModelingValentin De Bortoli, James Thornton, Jeremy Heng, Arnaud DoucetNeurIPS 2021 · 811 citations
Related papers
- Truncated Consistency ModelsSangyun Lee, Yilun Xu, Tomas Geffner, Giulia Fanti et al.ICLR 2025
- How to build a consistency model: Learning flow maps via self-distillationNicholas M. Boffi, Michael S. Albergo, Eric Vanden-EijndenNeurIPS 2025 · 111 citations
- Nearest Neighbour Score Estimators for Diffusion Generative ModelsMatthew Niedoba, Dylan Green, Saeid Naderiparizi, Vasileios Lioutas et al.ICML 2024 · 9 citations
- SCoT: Unifying Consistency Models and Rectified Flows via Straight-Consistent TrajectoriesZhangkai Wu, Xuhui Fan, Hongyu Wu, Longbing CaoNeurIPS 2025 · 12 citations
- Theory of Consistency Diffusion Models: Distribution Estimation Meets Fast SamplingZehao Dou, Minshuo Chen, Mengdi Wang, Zhuoran YangICML 2024 · 11 citations
