ACL Findings2025
StableToolBench-MirrorAPI: Modeling Tool Environments as Mirrors of 7, 000+ Real-World APIs
Zhicheng Guo, Sijie Cheng, Yuchen Niu, Hao Wang, Sicheng Zhou, Wenbing Huang, Yang Liu
摘要
The rapid advancement of large language models (LLMs) has spurred significant interest in tool learning, where LLMs are augmented with external tools to tackle complex tasks. However, existing tool environments face challenges in balancing stability, scalability, and realness, particularly for benchmarking purposes. To address this problem, we propose MirrorAPI, a novel framework that trains specialized LLMs to accurately simulate real API responses, effectively acting as "mirrors" to tool environments. Using a comprehensive dataset of request-response pairs from 7,000+ APIs, we employ supervised fine-tuning and chain-ofthought reasoning to enhance simulation fidelity. MirrorAPI achieves superior accuracy and stability compared to state-of-the-art methods, as demonstrated by its performance on the newly constructed MirrorAPI-Bench and its integration into StableToolBench.
问 Lune
尽管问,回答会引用原文。
Lune 会读完全文和相关的 Paper,每个回答都附上所依据的原文段落。
它引用的 Paper20
- BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger 等ICLR 2020 · 8,443 次引用
- Gorilla: Large Language Model Connected with Massive APIsShishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. GonzalezNeurIPS 2024 · 1,715 次引用
- WebShop: Towards Scalable Real-World Web Interaction with Grounded Language AgentsShunyu Yao, Howard Chen, John Yang, Karthik NarasimhanNeurIPS 2022 · 1,477 次引用
- WebArena: A Realistic Web Environment for Building Autonomous AgentsShuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou 等ICLR 2024 · 1,197 次引用
- CRITIC: Large Language Models Can Self-Correct with Tool-Interactive CritiquingZhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen 等ICLR 2024 · 699 次引用
相关 Paper
- StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language ModelsZhicheng Guo, Sijie Cheng, Hao Wang, Shihao Liang 等ACL Findings 2024 · 22 次引用
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu 等ICLR 2024 · 1,469 次引用
- Learning Evolving Tools for Large Language ModelsGuoxin Chen, Zhong Zhang, Xin Cong, Fangda Guo 等ICLR 2025
- Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection LearningZhiyuan Ma, Jiayu Liu, Xianzhen Luo, Zhenya Huang 等KDD 2025 · 4 次引用
- Do LLMs Catch Their Own Mistakes? A Comprehensive Benchmark for Reflective Tool Use LLMsZheyuan Liu, Liqiang Xiao, Yang Li, Hyokun Yun 等ACL Findings 2026
