Lune

ACL Findings2026

Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration

Jucheng Shen, Gaurav Sarkar, Yeonju Ro, Sharath Nittur Sridhar, Zhangyang Wang, Aditya Akella, Souvik Kundu

Abstract

We present CadLLM, a training-free method to accelerate the inference throughput of diffusion-based LLMs (dLLMs). We first investigate on the dynamic nature of the token unmasking confidence across blocks and steps. Based on this observation, we then present a lightweight adaptive approach that can control the generation block size, step size, and threshold based on the average confidence score of the unmasked tokens. We further reduce the softmaxing overhead of token probability generation by dynamically leveraging a subset of vocabulary size to regulate sampling breadth. CadLLM is a plug-and-play model-agnostic with KV caching based dLLMs. Extensive experiments on four popular tasks demonstrate the efficacy of CadLLM to yield throughput improvements ranging from 1.1-2.28× over the state-of-the-art baseline with competitive accuracy.

Code

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Learning Unmasking Policies for Diffusion Language Models cites Structured Denoising Diffusion Models in Discrete State-Spaces
  • Learning Unmasking Policies for Diffusion Language Models cites Large Language Diffusion Models
  • Learning Unmasking Policies for Diffusion Language Models cites Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
  • Large Language Diffusion Models cites Structured Denoising Diffusion Models in Discrete State-Spaces
  • Large Language Diffusion Models cites A Continuous Time Framework for Discrete Denoising Models
  • AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size cites Structured Denoising Diffusion Models in Discrete State-Spaces
  • AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size cites Large Language Diffusion Models
  • AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size cites Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
  • Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models cites Structured Denoising Diffusion Models in Discrete State-Spaces
  • Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models cites Large Language Diffusion Models
  • A Continuous Time Framework for Discrete Denoising Models cites Structured Denoising Diffusion Models in Discrete State-Spaces
  • Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding cites Structured Denoising Diffusion Models in Discrete State-Spaces
  • Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding cites Large Language Diffusion Models
  • Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding cites A Continuous Time Framework for Discrete Denoising Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Learning Unmasking Policies for Diffusion Language ModelsMetod Jazbec, Theo X. Olausson, Louis Béthune, Pierre Ablin et al.ICML 2026 · 24 citations
  2. Beyond Hard Masks: Progressive Token Evolution for Diffusion Language ModelsLinhao Zhong, Linyu Wu, Bozhen Fang, Tianjian Feng et al.ACL 2026 · 4 citations

Builds on5

  1. Structured Denoising Diffusion Models in Discrete State-SpacesJacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow et al.NeurIPS 2021 · 2,256 citations
  2. Large Language Diffusion ModelsShen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang et al.NeurIPS 2025 · 949 citations
  3. A Continuous Time Framework for Discrete Denoising ModelsAndrew Campbell, Joe Benton, Valentin De Bortoli, Thomas Rainforth et al.NeurIPS 2022 · 496 citations
  4. Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel DecodingChengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu et al.ICLR 2026 · 428 citations
  5. AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block SizeGuanxi Lu, Hao Mark Chen, Yuto Karashima, Zhican Wang et al.ICLR 2026 · 32 citations

Related papers

  1. ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-SkippingZijian Zhu, Fei Ren, Zhanhong Tan, Kaisheng MaICLR 2026 · 7 citations
  2. DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial AttentionYounjoo Lee, Seungkyun Dan, Junghoo Lee, Jaiyoung Park et al.ICML 2026 · 2 citations
  3. Attention Is All You Need for KV Cache in Diffusion LLMsQuan Nguyen-Tri, Mukul Ranjan, Zhiqiang ShenICLR 2026 · 36 citations
  4. d2^2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive CachingYuchu Jiang, Yue Cai, Xiangzhong Luo, Jiale Fu et al.ICLR 2026 · 16 citations
  5. FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided DiffusionZhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S. Abdelfattah et al.ICLR 2026 · 56 citations
Dusk over the sea between two cliffs drawn in fine vertical lines