ACL Findings2025
Tool learning via Inference-time Scaling and Cycle Verifier
Xiaobo Liang, Wenjin Xie, Juntao Li, Wanfu Wang, Yibin Chen, Kehai Chen, Min Zhang
Abstract
In inference-time scaling, Chain-of-Thought (CoT) plays a crucial role in enabling large language models (LLMs) to exhibit reasoning capabilities. However, in many scenarios, high-quality CoT data is scarce or even unavailable. In such cases, STaR-like methods can help LLMs synthesize CoT based on user queries and response, but they inevitably suffer from the risk of compounding errors. In this work, we tackle an even more challenging scenario: tool learning in the absence of user queries. We design a data scaling method using back-translation, which establishes an inference cycle to synthesize both user queries and CoT data. To reudce the compounding error of inference time, we introduce two rule-based verifiers to assess the validity of the synthesized CoT data. In particular, the Cycle Verifier facilitates performance improvement by continuously accumulating new data over multiple iterations. Our approach achieves a 75.4% pass rate and a 79.6% win rate using small models (7B) in StableToolBench. Notably, these results are obtained exclusively from self-synthesized high-quality data, without relying on external supervision or expert trajectories for warm-up.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Gorilla: Large Language Model Connected with Massive APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
- Gorilla: Large Language Model Connected with Massive APIs cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- Reflexion: language agents with verbal reinforcement learning cites Toolformer: Language Models Can Teach Themselves to Use Tools
- Reflexion: language agents with verbal reinforcement learning cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Gorilla: Large Language Model Connected with Massive APIs
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Reflexion: language agents with verbal reinforcement learning
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Cited by top-tier papers1
Builds on16
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
- Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
- Gorilla: Large Language Model Connected with Massive APIsShishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. GonzalezNeurIPS 2024 · 1,715 citations
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu et al.ICLR 2024 · 1,469 citations
Related papers
- A Joint Optimization Framework for Enhancing Efficiency of Tool Utilization in LLM AgentsBin Wu, Edgar Meij, Emine YilmazACL Findings 2025 · 1 citation
- CoT-based Synthesizer: Enhancing LLM Performance through Answer SynthesisBohan Zhang, Xiaokang Zhang, Jing Zhang, Jifan Yu et al.ACL 2025 · 8 citations
- Quality Matters: Evaluating Synthetic Data for Tool-Using LLMsShadi Iskander, Sofia Tolmach, Ori Shapira, Nachshon Cohen et al.EMNLP 2024 · 2 citations
- Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning CatalystHongru Wang, Deng Cai, Wanjun Zhong, Shijue Huang et al.ACL Findings 2025
- Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User IntentsZiyi Wang, Yuxuan Lu, Yimeng Zhang, Pei Chen et al.ACL 2026 · 9 citations
