ACL Findings2026
Chain-in-Tree: Back to Sequential Reasoning in LLM Tree Search
Abstract
Test-time scaling improves large language models (LLMs) on long-horizon reasoning tasks by allocating more compute at inference. LLM inference via tree search (LITS) achieves strong performance but is highly inefficient. We propose Chain-in-Tree (CiT), a plug-in framework that decides when to branch during search instead of expanding at every step. CiT introduces lightweight Branching Necessity (BN) evaluations, including BN-DP (direct prompting) and BN-SC (self-consistency). Integrated into Tree of Thoughts, ReST-MCTS, and RAP, BN-DP reduces token generation, model calls, and runtime by 75-85% on GSM8K and Math500, with often negligible or no accuracy loss. BN-SC typically yields substantial savings (up to 80%) generally but shows instability in 1-4 out of 14 settings, caused by a small subset of examples that produce extremely long reasoning steps. We theoretically prove that BN-DP never increases policy invocations and release unified implementations applicable across LITS frameworks. The full codebase is publicly available at https: //github.com/xinzhel/chain_in_tree.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models cites Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models cites Reasoning with Language Model is Planning with World Model
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Reasoning with Language Model is Planning with World Model
- Reasoning with Language Model is Planning with World Model cites Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- Reasoning with Language Model is Planning with World Model cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Don't Get Lost in the Trees: Streamlining LLM Reasoning by Overcoming Tree Search Exploration Pitfalls cites Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- Don't Get Lost in the Trees: Streamlining LLM Reasoning by Overcoming Tree Search Exploration Pitfalls cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
Builds on9
- Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
- Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language ModelsAndy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang et al.ICML 2024 · 443 citations
- Reasoning with Language Model is Planning with World ModelShibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong et al.EMNLP 2023 · 109 citations
- Don't Get Lost in the Trees: Streamlining LLM Reasoning by Overcoming Tree Search Exploration PitfallsAnte Wang, Linfeng Song, Ye Tian, Dian Yu et al.ACL 2025 · 9 citations
Related papers
- Slim-SC: Thought Pruning for Efficient Scaling with Self-ConsistencyColin Hong, Xu Guo, Anand Chaanan Singh, Esha Choukse et al.EMNLP 2025
- Thought calibration: Efficient and confident test-time scalingMenghua Wu, Cai Zhou, Stephen Bates, Tommi S. JaakkolaEMNLP 2025 · 12 citations
- Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking TokensWei-Lin Chen, Liqian Peng, Tian Tan, Chao Zhao et al.ICML 2026 · 20 citations
- Dynamic Early Exit in Reasoning ModelsChenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu et al.ICLR 2026 · 250 citations
- Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought ReasoningRenliang Sun, Wei Cheng, Dawei Li, Haifeng Chen et al.ACL 2026 · 11 citations
