ACL Findings2026
MemTR: Enhancing Tool-Calling Reliability via Uncertainty-Triggered FFN-Space Retracing
Hongtao Duan, Lu Jiang, Minying Zhang, Xiaobing Zhu, Tianpeng Bu, Hao Jiang, Xinyu Wei, Lulu Hu
Abstract
Tool calling requires Large Language Models (LLMs) to generate structured decisions including tool names and schema-constrained arguments, where small decoding mistakes can cause hard failures.Existing methods either rely on costly tool-use training data or only constrain syntax, leaving tool selection and argument value errors largely unsolved.We analyze tool calling failures through a Where-When lens: (Where) failures correlate with persistent uncertainty in late transformer layers, (When) uncertainty concentrates on content-bearing tokens (tool names and argument values) rather than schema tokens.Based on this, and motivated by evidence that transformer Feed Forward Networks (FFNs) act as key-value style memories that store and retrieve factual or associative mappings, we propose Memory Space Tool Retracing (MemTR), a weight-free decoding-time method that retrieves relevant tool evidence from the tool library and mixes it into the FFN-output at the uncertain layer, treating FFNs as key-value memories.Through extensive experiments on various model families (Qwen, Llama, and xLAM) and benchmarks (BFCL, ACEBench, APIBank), MemTR reduces tool calling failures by 2%-9% with only 1%-2% runtime overhead, without any fine-tuning or additional tool-use training data.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Reflexion: language agents with verbal reinforcement learning cites Toolformer: Language Models Can Teach Themselves to Use Tools
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Reflexion: language agents with verbal reinforcement learning
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
Builds on17
- Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
- Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
- The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu et al.ICLR 2024 · 1,469 citations
- ReTool: Reinforcement Learning for Strategic Tool Use in LLMsJiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang et al.ICLR 2026 · 406 citations
Related papers
- Self-Guided Function Calling in Large Language Models via Stepwise Experience RecallSijia Cui, Aiyao He, Shuai Xu, Hongming Zhang et al.EMNLP Findings 2025 · 2 citations
- MemoryLLM: Plug-n-Play Interpretable Feed-Forward Memory for TransformersAjay Jaiswal, Lauren Hannah, Han-Byul Kim, Duc Hoang et al.ICML 2026 · 3 citations
- Dynamic Tool Dependency Retrieval for Lightweight Function CallingBhrij Patel, Davide Belli, Amir Jalalirad, Maximilian Arnold et al.ACL Findings 2026
- Memory-augmented Query Reconstruction for LLM-based Knowledge Graph ReasoningMufan Xu, Gewen Liang, Kehai Chen, Wei Wang et al.ACL Findings 2025 · 2 citations
- ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function CallingJianghao Lin, Yuanyuan Shi, Xin Peng, Renjie Ding et al.ACL 2026 · 3 citations
