ACL Findings2026
PURE: Post-hoc Unlocking and REfinement for Discrete Diffusion Decoding
Yangryeol Park, Kunhui Lee, Hanback Choi, Cheoneum Park, Donghyeon Jeon, Inho Kang, Seung-Hoon Na
Abstract
Masked diffusion language models (MDLMs) enable efficient parallel decoding but are limited by a monotonic unmasking policy, where committed tokens cannot be revised.While remasking-based methods mitigate early errors, they mainly intervene during generation.In this work, we study post-hoc refinement of a completed draft and find that naive correction often fails because of contextual lock-in, a phenomenon in which local error patterns become self-reinforcing.To address this, we propose PURE (Post-hoc Unlocking and REfinement), a training-free inference algorithm for two-phase decoding.PURE profiles confidence dynamics during drafting to identify unstable regions via an instability score (∆ i ), then unlocks them through deterministic window masking and stochastic leftward relaxation.On reasoning benchmarks, PURE substantially improves accuracy when applied to LLaDA-8B-Instruct, including a gain of +12.9 points over the baseline on GSM8K.These gains require only a small refinement budget, yielding a favorable compute-quality trade-off for discrete diffusion decoding.A Results on LLaDA-1.5
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- High-Resolution Image Synthesis with Latent Diffusion Models cites Denoising Diffusion Probabilistic Models
- Structured Denoising Diffusion Models in Discrete State-Spaces cites Denoising Diffusion Probabilistic Models
- Large Language Diffusion Models cites Structured Denoising Diffusion Models in Discrete State-Spaces
- Large Language Diffusion Models cites Denoising Diffusion Probabilistic Models
- Video Diffusion Models cites Denoising Diffusion Probabilistic Models
Builds on21
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Video Diffusion ModelsJonathan Ho, Tim Salimans, Alexey A. Gritsenko, William Chan et al.NeurIPS 2022 · 2,948 citations
- Structured Denoising Diffusion Models in Discrete State-SpacesJacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow et al.NeurIPS 2021 · 2,256 citations
- Large Language Diffusion ModelsShen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang et al.NeurIPS 2025 · 949 citations
Related papers
- CORE: Context-Robust Remasking for Diffusion Language ModelsKevin Zhai, Sabbir Mollah, Zhenyi Wang, Mubarak ShahICML 2026 · 10 citations
- Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMsFeng Hong, Geng Yu, Yushi Ye, Haicheng Huang et al.ICLR 2026 · 48 citations
- Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked DiffusionsJaeyeon Kim, Kulin Shah, Vasilis Kontonis, Sham M. Kakade et al.ICML 2025
- Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language ModelsMingyu Cao, Alvaro Correia, Christos Louizos, Shiwei Liu et al.ICML 2026
- Lookahead Unmasking Elicits Reliable Decoding in Diffusion Language ModelsSanghyun Lee, Seungryong Kim, Jongho Park, Dongmin ParkICML 2026 · 13 citations
