Lune

ACL Findings2026

MemTR: Enhancing Tool-Calling Reliability via Uncertainty-Triggered FFN-Space Retracing

Hongtao Duan, Lu Jiang, Minying Zhang, Xiaobing Zhu, Tianpeng Bu, Hao Jiang, Xinyu Wei, Lulu Hu

Abstract

Tool calling requires Large Language Models (LLMs) to generate structured decisions including tool names and schema-constrained arguments, where small decoding mistakes can cause hard failures.Existing methods either rely on costly tool-use training data or only constrain syntax, leaving tool selection and argument value errors largely unsolved.We analyze tool calling failures through a Where-When lens: (Where) failures correlate with persistent uncertainty in late transformer layers, (When) uncertainty concentrates on content-bearing tokens (tool names and argument values) rather than schema tokens.Based on this, and motivated by evidence that transformer Feed Forward Networks (FFNs) act as key-value style memories that store and retrieve factual or associative mappings, we propose Memory Space Tool Retracing (MemTR), a weight-free decoding-time method that retrieves relevant tool evidence from the tool library and mixes it into the FFN-output at the uncertain layer, treating FFNs as key-value memories.Through extensive experiments on various model families (Qwen, Llama, and xLAM) and benchmarks (BFCL, ACEBench, APIBank), MemTR reduces tool calling failures by 2%-9% with only 1%-2% runtime overhead, without any fine-tuning or additional tool-use training data.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Reflexion: language agents with verbal reinforcement learning cites Toolformer: Language Models Can Teach Themselves to Use Tools
  • ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Reflexion: language agents with verbal reinforcement learning
  • ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on17

  1. Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
  2. Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
  3. The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations
  4. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu et al.ICLR 2024 · 1,469 citations
  5. ReTool: Reinforcement Learning for Strategic Tool Use in LLMsJiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang et al.ICLR 2026 · 406 citations

Related papers

  1. Self-Guided Function Calling in Large Language Models via Stepwise Experience RecallSijia Cui, Aiyao He, Shuai Xu, Hongming Zhang et al.EMNLP Findings 2025 · 2 citations
  2. MemoryLLM: Plug-n-Play Interpretable Feed-Forward Memory for TransformersAjay Jaiswal, Lauren Hannah, Han-Byul Kim, Duc Hoang et al.ICML 2026 · 3 citations
  3. Dynamic Tool Dependency Retrieval for Lightweight Function CallingBhrij Patel, Davide Belli, Amir Jalalirad, Maximilian Arnold et al.ACL Findings 2026
  4. Memory-augmented Query Reconstruction for LLM-based Knowledge Graph ReasoningMufan Xu, Gewen Liang, Kehai Chen, Wei Wang et al.ACL Findings 2025 · 2 citations
  5. ToolPRM: Fine-Grained Inference Scaling of Structured Outputs for Function CallingJianghao Lin, Yuanyuan Shi, Xin Peng, Renjie Ding et al.ACL 2026 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines