Lune

ICML2025Top-tier venue

Improving Consistency Models with Generator-Augmented Flows

Thibaut Issenhuth, Sangchul Lee, Ludovic Dos Santos, Jean-Yves Franceschi, Chansoo Kim, Alain Rakotomamonjy

Abstract

Consistency models imitate the multi-step sampling of score-based diffusion in a single forward pass of a neural network. They can be learned in two ways: consistency distillation and consistency training. The former relies on the true velocity field of the corresponding differential equation, approximated by a pre-trained neural network. In contrast, the latter uses a single-sample Monte Carlo estimate of this velocity field. The related estimation error induces a discrepancy between consistency distillation and training that, we show, still holds in the continuous-time limit. To alleviate this issue, we propose a novel flow that transports noisy data towards their corresponding outputs derived from a consistency model. We prove that this flow reduces the previously identified discrepancy and the noise-data transport cost. Consequently, our method not only accelerates consistency training convergence but also enhances its overall performance. The code is available at: github.com/thibautissenhuth/consistency GC.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Elucidating the Design Space of Diffusion-Based Generative Models cites Denoising Diffusion Probabilistic Models
  • Elucidating the Design Space of Diffusion-Based Generative Models cites Score-Based Generative Modeling through Stochastic Differential Equations
  • VCT: Training Consistency Models with Variational Noise Coupling cites Elucidating the Design Space of Diffusion-Based Generative Models
  • VCT: Training Consistency Models with Variational Noise Coupling cites Consistency Models
  • VCT: Training Consistency Models with Variational Noise Coupling cites Denoising Diffusion Probabilistic Models
  • VCT: Training Consistency Models with Variational Noise Coupling cites Score-Based Generative Modeling through Stochastic Differential Equations
  • Consistency Models cites Elucidating the Design Space of Diffusion-Based Generative Models
  • Consistency Models cites Denoising Diffusion Probabilistic Models
  • Consistency Models cites Score-Based Generative Modeling through Stochastic Differential Equations
  • Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling cites Denoising Diffusion Probabilistic Models
  • Score-Based Generative Modeling through Stochastic Differential Equations cites Denoising Diffusion Probabilistic Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. VCT: Training Consistency Models with Variational Noise CouplingGianluigi Silvestri, Luca Ambrogioni, Chieh-Hsin Lai, Yuhta Takida et al.ICML 2025

Builds on19

  1. Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
  2. Elucidating the Design Space of Diffusion-Based Generative ModelsTero Karras, Miika Aittala, Timo Aila, Samuli LaineNeurIPS 2022 · 3,959 citations
  3. Consistency ModelsYang Song, Prafulla Dhariwal, Mark Chen, Ilya SutskeverICML 2023 · 1,720 citations
  4. Score-Based Generative Modeling through Stochastic Differential EquationsYang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar et al.ICLR 2021 · 1,270 citations
  5. Diffusion Schrödinger Bridge with Applications to Score-Based Generative ModelingValentin De Bortoli, James Thornton, Jeremy Heng, Arnaud DoucetNeurIPS 2021 · 811 citations

Related papers

  1. Truncated Consistency ModelsSangyun Lee, Yilun Xu, Tomas Geffner, Giulia Fanti et al.ICLR 2025
  2. How to build a consistency model: Learning flow maps via self-distillationNicholas M. Boffi, Michael S. Albergo, Eric Vanden-EijndenNeurIPS 2025 · 111 citations
  3. Nearest Neighbour Score Estimators for Diffusion Generative ModelsMatthew Niedoba, Dylan Green, Saeid Naderiparizi, Vasileios Lioutas et al.ICML 2024 · 9 citations
  4. SCoT: Unifying Consistency Models and Rectified Flows via Straight-Consistent TrajectoriesZhangkai Wu, Xuhui Fan, Hongyu Wu, Longbing CaoNeurIPS 2025 · 12 citations
  5. Theory of Consistency Diffusion Models: Distribution Estimation Meets Fast SamplingZehao Dou, Minshuo Chen, Mengdi Wang, Zhuoran YangICML 2024 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines