Error Analysis of Discrete Flow with Generator Matching
Zhengyan Wan, Yidong Ouyang, Qiang Yao, Liyan Xie, Fang Fang, Hongyuan Zha, Guang Cheng
摘要
Discrete flow models offer a powerful framework for learning distributions over discrete state spaces and have demonstrated superior performance compared to the discrete diffusion models. However, their convergence properties and error analysis remain largely unexplored. In this work, we develop a unified framework grounded in stochastic calculus theory to systematically investigate the theoretical properties of discrete flow models. Specifically, by leveraging a Girsanov-type theorem for the path measures of two continuous-time Markov chains (CTMCs), we present a comprehensive error analysis that accounts for both transition rate estimation error and early stopping error. In fact, the estimation error of transition rates has received little attention in existing works. Unlike discrete diffusion models, discrete flow incurs no initialization error caused by truncating the time horizon in the noising process. Building on generator matching and uniformization, we establish non-asymptotic error bounds for distribution estimation without the boundedness condition on oracle transition rates. Furthermore, we derive a faster rate of total variation convergence for the estimated distribution with the boundedness condition, yielding a nearly optimal rate in terms of sample size. Our results provide the first error analysis for discrete flow models. We also investigate model performance under different settings based on simulation results.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper2
- A Time-Reparameterized Cumulative Intensity Extrapolation Sampler for Discrete Flow MatchingFeiyang Fu, Hehe FanICML 2026
- Corrected Samplers for Discrete Flow ModelsZhengyan Wan, Yidong Ouyang, Liyan Xie, Hongyuan Zha 等ICML 2026
它引用的顶会 Paper30
- Structured Denoising Diffusion Models in Discrete State-SpacesJacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow 等NeurIPS 2021 · 被引用 2,256 次
- Large Language Diffusion ModelsShen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang 等NeurIPS 2025 · 被引用 949 次
- Simple and Effective Masked Diffusion Language ModelsSubham S. Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan 等NeurIPS 2024 · 被引用 929 次
- Argmax Flows and Multinomial Diffusion: Learning Categorical DistributionsEmiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré 等NeurIPS 2021 · 被引用 782 次
- Simplified and Generalized Masked Diffusion for Discrete DataJiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet 等NeurIPS 2024 · 被引用 693 次
相关 Paper
- Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time AnalysisZikun Zhang, Zixiang Chen, Quanquan GuICLR 2025
- How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral FrameworkYinuo Ren, Haoxuan Chen, Grant M. Rotskoff, Lexing YingICLR 2025
- Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion ModelsYuchen Liang, Renxiang Huang, Lifeng Lai, Ness Shroff 等NeurIPS 2025 · 被引用 19 次
- High-Order Flow Matching: Unified Framework and Sharp Statistical RatesMaojiang Su, Jerry Yao-Chieh Hu, Yi-Chen Lee, Ning Zhu 等NeurIPS 2025 · 被引用 9 次
- Theoretical guarantees in KL for Diffusion Flow MatchingMarta Gentiloni Silveri, Alain Durmus, Giovanni ConfortiNeurIPS 2024 · 被引用 16 次
