Lune

EMNLP Findings2024

Enable Fast Sampling for Seq2Seq Text Diffusion

Pan Liu, Xiaohua Tian, Zhouhan Lin

Abstract

Diffusion models exhibit promising capacity for generating high-quality text. However, owing to the curved nature of the generation path, they necessitate traversing numerous steps to guarantee high text quality. In this paper, we propose an efficient model FMSeq 1 , which utilizes flow matching to straighten the generation path, thereby enabling fast sampling for diffusion-based seq2seq text generation. Specifically, we construct transport flow only on the target sequences to adapt the diffusionbased model to flow matching. Furthermore, we explore different settings and identify targetparameterization, self-conditioning, and timedifference as three effective techniques to improve the generation quality under a few steps. Experiments on four popular tasks demonstrate that FMSeq generates texts of comparable quality to the SOTA diffusion-based DiffuSeq in just 10 steps, achieving a 200-fold speedup.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • High-Resolution Image Synthesis with Latent Diffusion Models cites Diffusion Models Beat GANs on Image Synthesis
  • High-Resolution Image Synthesis with Latent Diffusion Models cites Denoising Diffusion Probabilistic Models
  • Elucidating the Design Space of Diffusion-Based Generative Models cites High-Resolution Image Synthesis with Latent Diffusion Models
  • Elucidating the Design Space of Diffusion-Based Generative Models cites Diffusion Models Beat GANs on Image Synthesis
  • Elucidating the Design Space of Diffusion-Based Generative Models cites Denoising Diffusion Probabilistic Models
  • Diffusion Models Beat GANs on Image Synthesis cites Denoising Diffusion Probabilistic Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. FAVE: Flow-based Average Velocity Establishment for Sequential RecommendationKe Shi, Yao Zhang, Feng Guo, Jinyuan Zhang et al.SIGIR 2026

Builds on19

  1. Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
  2. Diffusion Models Beat GANs on Image SynthesisPrafulla Dhariwal, Alexander Quinn NicholNeurIPS 2021 · 13,211 citations
  3. High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
  4. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  5. Elucidating the Design Space of Diffusion-Based Generative ModelsTero Karras, Miika Aittala, Timo Aila, Samuli LaineNeurIPS 2022 · 3,959 citations

Related papers

  1. FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language ModelsAmin Karimi Monsefi, Nikhil Bhendawade, Manuel Rafael Ciosici, Dominic Culver et al.ICLR 2026 · 15 citations
  2. XYZFlow: Scaling Multidimensional Shortcut Flows for Efficient Generative ModelingJinxiu Liu, Xuanming Liu, Kangfu Mei, Yandong Wen et al.ICML 2026
  3. DiffuSeq: Sequence to Sequence Text Generation with Diffusion ModelsShansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu et al.ICLR 2023 · 94 citations
  4. SimGFM: Simplifying Discrete Flow Matching for Graph GenerationChunyu Luo, Yuankai Luo, Xiao-Ming Wu, Lei ShiICML 2026
  5. Multistep Distillation of Diffusion Models via Moment MatchingTim Salimans, Thomas Mensink, Jonathan Heek, Emiel HoogeboomNeurIPS 2024 · 93 citations
Dusk over the sea between two cliffs drawn in fine vertical lines