Masked Diffusion Models as Energy Minimization
Sitong Chen, Shen Nie, Jiacheng Sun, Zijin Feng, Zhenguo Li, Ji-Rong Wen, Chongxuan Li
摘要
We present a systematic theoretical framework that interprets masked diffusion models (MDMs) as solutions to energy minimization problems in discrete optimal transport. Specifically, we prove that three distinct energy formulations--kinetic, conditional kinetic, and geodesic energy--are mathematically equivalent under the structure of MDMs, and that MDMs minimize all three when the mask schedule satisfies a closed-form optimality condition. This unification not only clarifies the theoretical foundations of MDMs, but also motivates practical improvements in sampling. By parameterizing interpolation schedules via Beta distributions, we reduce the schedule design space to a tractable 2D search, enabling efficient post-training tuning without model modification. Experiments on synthetic and real-world benchmarks demonstrate that our energy-inspired schedules outperform hand-crafted baselines, particularly in low-step sampling settings.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper1
问问它们各自怎么用它它引用的顶会 Paper28
- Structured Denoising Diffusion Models in Discrete State-SpacesJacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow 等NeurIPS 2021 · 被引用 2,256 次
- Solving Quantitative Reasoning Problems with Language ModelsAitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer 等NeurIPS 2022 · 被引用 2,039 次
- Large Language Diffusion ModelsShen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang 等NeurIPS 2025 · 被引用 949 次
- Simple and Effective Masked Diffusion Language ModelsSubham S. Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan 等NeurIPS 2024 · 被引用 929 次
- Muse: Text-To-Image Generation via Masked Generative TransformersHuiwen Chang, Han Zhang, Jarred Barber, Aaron Maschinot 等ICML 2023 · 被引用 751 次
相关 Paper
- Score-Optimal Diffusion SchedulesChristopher Williams, Andrew Campbell, Arnaud Doucet, Saifuddin SyedNeurIPS 2024 · 被引用 19 次
- Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference PoliciesChunsan Hong, Seonho An, Min-Soo Kim, Jong Chul YeICLR 2026 · 被引用 23 次
- Align Your Steps: Optimizing Sampling Schedules in Diffusion ModelsAmirmojtaba Sabour, Sanja Fidler, Karsten KreisICML 2024 · 被引用 74 次
- Conditional Diffusion SamplingFrancisco M Castro-Macías, Pablo Morales-Alvarez, Saifuddin Syed, Daniel Hernández-Lobato 等ICML 2026 · 被引用 7 次
- Flow Matching with General Discrete Paths: A Kinetic-Optimal PerspectiveNeta Shaul, Itai Gat, Marton Havasi, Daniel Severo 等ICLR 2025
