ACL Findings2026
Training Verifier to Assessing Complex Real-World Tool-Use Trajectories
Linzhuang Sun, Mingyang Chen, Hao Liang, Tianpeng Li, Yijie Zhou, Chenzheng Zhu, Tianyu Guo, Huanyao Zhang, Jingxuan Wei, Bihui Yu, Fan Yang, Wentao Zhang
Abstract
Training effective AI agents for real-world tooluse interactions requires data that faithfully captures the dynamics of human-agent collaboration. However, such data is scarce, and existing methods often resort to synthetic data generation. The inherently dynamic and complex nature of user-agent interactions makes ensuring data quality particularly challenging. Current verification approaches are typically entangled with the synthesis process itself, resulting in complicated implementations that undermine both reproducibility and scalability. To address this, we introduce Tool-Verifier-7B, a plug-and-play framework for data quality control in tool-use scenarios. Building on this verifier and our data synthesis strategy, we construct the Tool-Verify dataset, which contains 3,295 curated samples. To directly assess verifier performance, we further release Tool-V-Bench, a benchmark of 165 humanvalidated trajectories spanning diverse interaction complexities. Comprehensive experiments show that Tool-Verifier-7B surpasses GPT-4o on Tool-V-Bench. Moreover, the Tool-Verify cleaned dataset achieves superior performance compared to the previous APIGen-MT dataset.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Gorilla: Large Language Model Connected with Massive APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs cites Gorilla: Large Language Model Connected with Massive APIs
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs cites Toolformer: Language Models Can Teach Themselves to Use Tools
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Gorilla: Large Language Model Connected with Massive APIs
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
Builds on11
- Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
- Gorilla: Large Language Model Connected with Massive APIsShishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. GonzalezNeurIPS 2024 · 1,715 citations
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu et al.ICLR 2024 · 1,469 citations
- ReTool: Reinforcement Learning for Strategic Tool Use in LLMsJiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang et al.ICLR 2026 · 406 citations
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMsMinghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song et al.EMNLP 2023 · 72 citations
Related papers
- ToolACE: Winning the Points of LLM Function CallingWeiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao et al.ICLR 2025
- Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection LearningZhiyuan Ma, Jiayu Liu, Xianzhen Luo, Zhenya Huang et al.KDD 2025 · 4 citations
- Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool UsageZhi Gao, Bofei Zhang, Pengxiang Li, Xiaojian Ma et al.ICLR 2025
- Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User IntentsZiyi Wang, Yuxuan Lu, Yimeng Zhang, Pei Chen et al.ACL 2026 · 9 citations
- TOOLVERIFIER: Generalization to New Tools via Self-VerificationDheeraj Mekala, Jason Weston, Jack Lanchantin, Roberta Raileanu et al.EMNLP Findings 2024 · 6 citations
