ACL Findings2025
Entropy-based Exploration Conduction for Multi-step Reasoning
Jinghan Zhang, Xiting Wang, Fengran Mo, Yeyang Zhou, Wanfu Gao, Kunpeng Liu
Abstract
Multi-step processes via large language models (LLMs) have proven effective for solving complex reasoning tasks. However, the depth of exploration of the reasoning procedure can significantly affect the task performance. Existing methods to automatically decide the depth often lead to high cost and a lack of flexibility. To address these issues, we propose Entropybased Exploration Depth Conduction (Entroduction), a novel method that dynamically adjusts the exploration depth during multi-step reasoning by monitoring LLM's output entropy and variance entropy. We employ these two features to capture the model's uncertainty of the current step and the fluctuation of uncertainty across consecutive reasoning steps. Based on the observed entropy changes, the LLM selects whether to deepen, expand, or stop exploration according to the probability, which facilitates the trade-off between the reasoning accuracy and exploration effectiveness. Experimental results across four benchmark datasets demonstrate the efficacy of Entro-duction.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
12 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- RAST: Reasoning Activation in LLMs via Small-model Transfer cites Let's Verify Step by Step
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Language Models are Few-Shot Learners
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Language Models are Few-Shot Learners
- Self-Consistency Improves Chain of Thought Reasoning in Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Let's Verify Step by Step cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Let's Verify Step by Step cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- PEAR: Phase Entropy Aware Reward for Efficient Reasoning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understanding cites SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning
Cited by top-tier papers12
- SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement LearningZhongwei Wan, Zhihao Dou, Che Liu, Yu Zhang et al.NeurIPS 2025 · 63 citations
- UniConv: Unifying Retrieval and Response Generation for Large Language Models in ConversationsFengran Mo, Yifan Gao, Chuan Meng, Xin Liu et al.ACL 2025 · 22 citations
- REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video UnderstandingJiaze Li, Hao Yin, Wenhui Tan, Jingyang Chen et al.CVPR 2026 · 14 citations
- RAST: Reasoning Activation in LLMs via Small-model TransferSiru Ouyang, Xinyu Zhu, Zilin Xiao, Minhao Jiang et al.NeurIPS 2025 · 9 citations
- PEAR: Phase Entropy Aware Reward for Efficient ReasoningChen Huang, Wei Lu, Wenxuan ZhangICLR 2026 · 7 citations
Builds on24
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
- Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
Related papers
- INFORM : Information eNtropy based multi-step reasoning FOR large language ModelsChuyue Zhou, Wangjie You, Juntao Li, Jing Ye et al.EMNLP 2023 · 2 citations
- Reasoning with Exploration: An Entropy PerspectiveDaixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai et al.AAAI 2026 · 216 citations
- Reinforced Efficient Reasoning via Semantically Diverse ExplorationZiqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang et al.ACL 2026 · 5 citations
- Restoring Exploration after Post-Training: Latent Exploration Decoding for Large Reasoning ModelsWenhui Tan, Fiorenzo Parascandolo, Enver Sangineto, Jianzhong Ju et al.ICML 2026 · 2 citations
- Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement LearningXiaoyun Zhang, Xiaojian Yuan, Di Huang, Wang You et al.ACL Findings 2026
