Lune

ACL Findings2026

Towards Self-Improving Error Diagnosis in Multi-Agent Systems

Jiazheng Li, Emine Yilmaz, Bei Chen, Thu Le

Abstract

Large Language Model (LLM)-based Multi-Agent Systems (MAS) enable complex problem-solving but introduce significant debugging challenges, characterized by long interaction traces, inter-agent dependencies, and delayed error manifestation. Existing diagnostic approaches often rely on expensive expert annotation or "LLM-as-a-judge" paradigms, which struggle to pinpoint decisive error steps within extended contexts. In this paper, we introduce ERRORPROBE, a self-improving framework for semantic failure attribution that identifies responsible agents and the originating error step. The framework operates via a three-stage pipeline:

(1) operationalizing the MAS failure taxonomy to detect local anomalies, (2) performing symptom-driven backward tracing to prune irrelevant context, and (3) employing a specialized multi-agent team (Strategist, Investigator, Arbiter) to validate error hypotheses through tool-grounded execution. Crucially, ERRORPROBE maintains a verified episodic memory that updates only when error patterns are confirmed by executable evidence, without the need for annotation. Experiments across the TRACERTRAJ and WHO&WHEN benchmarks demonstrate that ERRORPROBE significantly outperforms baselines, particularly in step-level localization, while the verified memory enables robust cross-domain transfer without retraining.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Gorilla: Large Language Model Connected with Massive APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
  • Reflexion: language agents with verbal reinforcement learning cites Self-Refine: Iterative Refinement with Self-Feedback
  • Reflexion: language agents with verbal reinforcement learning cites Generative Agents: Interactive Simulacra of Human Behavior
  • Reflexion: language agents with verbal reinforcement learning cites Toolformer: Language Models Can Teach Themselves to Use Tools
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on17

  1. Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
  2. Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
  3. Self-Refine: Iterative Refinement with Self-FeedbackAman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan et al.NeurIPS 2023 · 4,972 citations
  4. Generative Agents: Interactive Simulacra of Human BehaviorJoon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris et al.UIST 2023 · 1,882 citations
  5. Gorilla: Large Language Model Connected with Massive APIsShishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. GonzalezNeurIPS 2024 · 1,715 citations

Related papers

  1. Scope Delineation Before Localization: A Two-Stage Framework for Enhancing Failure Attribution in Multi-Agent SystemsKai Sun, Wenqiang Li, Bo Dong, Yuxin Lin et al.AAAI 2026
  2. AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?Guibin Zhang, Junhao Wang, Junjie Chen, Wangchunshu Zhou et al.ICLR 2026 · 107 citations
  3. Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent SystemsMengzhuo Chen, Junjie Wang, Fangwen Mu, Yawen Wang et al.ACL 2026 · 5 citations
  4. Aegis: Automated Error Generation and Attribution for Multi-Agent SystemsFanqi Kong, Ruijie Zhang, Huaxiao Yin, Guibin Zhang et al.ICLR 2026 · 16 citations
  5. DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent SystemsMing Ma, Jue Zhang, Fangkai Yang, Yu Kang et al.ICLR 2026 · 24 citations
Dusk over the sea between two cliffs drawn in fine vertical lines