Lune

ACL Findings2026

Chain-in-Tree: Back to Sequential Reasoning in LLM Tree Search

Xinzhe Li

Abstract

Test-time scaling improves large language models (LLMs) on long-horizon reasoning tasks by allocating more compute at inference. LLM inference via tree search (LITS) achieves strong performance but is highly inefficient. We propose Chain-in-Tree (CiT), a plug-in framework that decides when to branch during search instead of expanding at every step. CiT introduces lightweight Branching Necessity (BN) evaluations, including BN-DP (direct prompting) and BN-SC (self-consistency). Integrated into Tree of Thoughts, ReST-MCTS, and RAP, BN-DP reduces token generation, model calls, and runtime by 75-85% on GSM8K and Math500, with often negligible or no accuracy loss. BN-SC typically yields substantial savings (up to 80%) generally but shows instability in 1-4 out of 14 settings, caused by a small subset of examples that produce extremely long reasoning steps. We theoretically prove that BN-DP never increases policy invocations and release unified implementations applicable across LITS frameworks. The full codebase is publicly available at https: //github.com/xinzhel/chain_in_tree.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models cites Tree of Thoughts: Deliberate Problem Solving with Large Language Models
  • Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models cites Reasoning with Language Model is Planning with World Model
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models cites Reasoning with Language Model is Planning with World Model
  • Reasoning with Language Model is Planning with World Model cites Tree of Thoughts: Deliberate Problem Solving with Large Language Models
  • Reasoning with Language Model is Planning with World Model cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Don't Get Lost in the Trees: Streamlining LLM Reasoning by Overcoming Tree Search Exploration Pitfalls cites Tree of Thoughts: Deliberate Problem Solving with Large Language Models
  • Don't Get Lost in the Trees: Streamlining LLM Reasoning by Overcoming Tree Search Exploration Pitfalls cites Self-Consistency Improves Chain of Thought Reasoning in Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on9

  1. Tree of Thoughts: Deliberate Problem Solving with Large Language ModelsShunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran et al.NeurIPS 2023 · 5,068 citations
  2. Self-Consistency Improves Chain of Thought Reasoning in Language ModelsXuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le et al.ICLR 2023 · 681 citations
  3. Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language ModelsAndy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang et al.ICML 2024 · 443 citations
  4. Reasoning with Language Model is Planning with World ModelShibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong et al.EMNLP 2023 · 109 citations
  5. Don't Get Lost in the Trees: Streamlining LLM Reasoning by Overcoming Tree Search Exploration PitfallsAnte Wang, Linfeng Song, Ye Tian, Dian Yu et al.ACL 2025 · 9 citations

Related papers

  1. Slim-SC: Thought Pruning for Efficient Scaling with Self-ConsistencyColin Hong, Xu Guo, Anand Chaanan Singh, Esha Choukse et al.EMNLP 2025
  2. Thought calibration: Efficient and confident test-time scalingMenghua Wu, Cai Zhou, Stephen Bates, Tommi S. JaakkolaEMNLP 2025 · 12 citations
  3. Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking TokensWei-Lin Chen, Liqian Peng, Tian Tan, Chao Zhao et al.ICML 2026 · 20 citations
  4. Dynamic Early Exit in Reasoning ModelsChenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu et al.ICLR 2026 · 250 citations
  5. Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought ReasoningRenliang Sun, Wei Cheng, Dawei Li, Haifeng Chen et al.ACL 2026 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines