Break the Block: Dynamic-size Reasoning Blocks for Diffusion Large Language Models via Monotonic Entropy Descent with Reinforcement Learning
Yan Jiang, Ruihong Qiu, Zi Huang
摘要
Recent diffusion large language models (dLLMs) have demonstrated both effectiveness and efficiency in reasoning via a block-based semiautoregressive generation paradigm. Despite their progress, the fixed-size block generations remain a critical bottleneck for effective and coherent reasoning. (i) From a global perspective, different reasoning tasks would correspond to different optimal decoding block sizes, which makes a "one-size-fits-all" assumption ineffective. (ii) Even within a single reasoning task, the rigid block partitioning would break the logical flow and reduce reasoning coherence. Through empirical observations, we reveal that for block-wise entropy, incorrect reasoning exhibits a fluctuating and unsteady trend between blocks, whereas the correctly generated tasks follow a consistent descending trend. Therefore, this paper proposes b1, a novel post-training framework for dLLMs that learns dynamic-size reasoning blocks via a Monotonic Entropy Descent objective with reinforcement learning to enhance reasoning coherence. b1 integrates seamlessly as a plug-and-play module with existing dLLM's post-training algorithms. Extensive experiments across various reasoning benchmarks showcase b1's consistent improvement over existing fixedsize block baselines. Our code has been released at https://github.com/YanJiangJerry/Block-R1 .
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper11
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida 等NeurIPS 2022 · 被引用 24,707 次
- Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards 等ICLR 2024 · 被引用 3,045 次
- FlashAttention-2: Faster Attention with Better Parallelism and Work PartitioningTri DaoICLR 2024 · 被引用 2,600 次
- Large Language Diffusion ModelsShen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang 等NeurIPS 2025 · 被引用 949 次
- LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion ModelsFengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang 等ACL 2026 · 被引用 229 次
相关 Paper
- d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement LearningSiyan Zhao, Devaansh Gupta, Qinqing Zheng, Aditya GroverNeurIPS 2025 · 被引用 191 次
- Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language ModelsJia Deng, Junyi Li, Xin Zhao, Jinpeng Wang 等ACL 2026
- Lavida-R1: Advancing Reasoning for Unified Multimodal Diffusion Language ModelsShufan Li, Yuchen Zhu, Kangning Liu, Zhe Lin 等ICML 2026 · 被引用 4 次
- Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy OptimizationYuchen Zhu, Wei Guo, Jaemoo Choi, Petr Molodyk 等ICML 2026 · 被引用 13 次
- Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language ModelsBoyan Han, Yiwei Wang, Yi Song, Yujun Cai 等ACL 2026
