Lune

EMNLP Findings2025

How Can Input Reformulation Improve Tool Usage Accuracy in a Complex Dynamic Environment? A Study on tau-bench

Venkatesh Mishra, Amir Saeidi, Satyam Raj, Mutsumi Nakamura, Gaowen Liu, Ali Payani, Jayanth Srinivasa, Chitta Baral

Abstract

Recent advances in reasoning and planning capabilities of large language models (LLMs) have enabled their potential as autonomous agents capable of tool use in dynamic environments. However, in multi-turn conversational environments like τ -bench, these agents often struggle with consistent reasoning, adherence to domain-specific policies, and extracting correct information over a long horizon of tool-calls and conversation. To capture and mitigate these failures, we conduct a comprehensive manual analysis of the common errors occurring in the conversation trajectories. We then experiment with reformulations of inputs to the tool-calling agent for improvement in agent decision-making. Finally, we propose the Input-Reformulation Multi-Agent (IRMA) framework, which automatically reformulates user queries augmented with relevant domain rules and tool suggestions for the tool-calling agent to focus on. The results show that IRMA significantly outperforms ReAct, Function Calling, and Self-Reflection by 16.1%, 12.7%, and 19.1%, respectively, in overall pass^5 scores. These findings highlight the superior reliability and consistency of IRMA compared to other methods in dynamic environments.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Gorilla: Large Language Model Connected with Massive APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
  • GAIA: a benchmark for General AI Assistants cites Gorilla: Large Language Model Connected with Massive APIs
  • ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Gorilla: Large Language Model Connected with Massive APIs
  • ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on18

  1. Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
  2. Gorilla: Large Language Model Connected with Massive APIsShishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. GonzalezNeurIPS 2024 · 1,715 citations
  3. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu et al.ICLR 2024 · 1,469 citations
  4. GAIA: a benchmark for General AI AssistantsGrégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun et al.ICLR 2024 · 716 citations
  5. ReTool: Reinforcement Learning for Strategic Tool Use in LLMsJiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang et al.ICLR 2026 · 406 citations

Related papers

  1. FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use EnvironmentsAmir Saeidi, Venkatesh Mishra, Souradeep Mukhopadhyay, Gaowen Liu et al.ACL Findings 2026
  2. Invocation Refiner: A Plug-and-Play Module for Rectifying LLM Tool InvocationsQirui Jiao, Dian Jiao, Nan Du, Ying Shen et al.ACL Findings 2026
  3. Failure makes the agent stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool InteractionsJunhao Su, Yuanliang Wan, Junwei Yang, Hengyu Shi et al.ACL Findings 2026 · 1 citation
  4. AutoTool: Efficient Tool Selection for Large Language Model AgentsJingyi Jia, Qinbin LiAAAI 2026 · 4 citations
  5. RefactorBench: Evaluating Stateful Reasoning in Language Agents Through CodeDhruv Gautam, Spandan Garg, Jinu Jang, Neel Sundaresan et al.ICLR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines