ACL Findings2026
Towards Self-Evolving Agents: Enabling Autonomy through Interactive Experience Refinement
Cheng Yang, Xuemeng Yang, Licheng Wen, Daocheng Fu, Jianbiao Mei, Rong Wu, Pinlong Cai, Yufan Shen, Nianchen Deng, Xu Jia, Botian Shi, Yu Qiao, Haifeng Li
Abstract
Large Language Models often struggle with complex, multi-step operational tasks because they remain static during inference and cannot learn from past experience. To address this, we propose MUSE, a framework that enables iterative self-improvement through a hierarchical Memory Module. MUSE organizes crossdomain insights to facilitate the orchestration of long-horizon workflows. The core of our approach is an autonomous post-execution critique mechanism: after completing each subtask, the system analyzes its operational logs and distills raw execution data into structured, reusable knowledge. This allows the agent to evolve dynamically rather than relying on fixed parameters. Evaluated on the rigorous TAC productivity benchmark, MUSE achieves new state-of-the-art results, significantly outperforming previous methods using only the streamlined Gemini-2.5 Flash model. Our analysis demonstrates that MUSE's performance scales with the accumulation of insights and exhibits strong cross-task transferability, marking a key step toward autonomous systems capable of lifelong learning in professional environments. Demo videos can be found in our supplementary materials.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Reflexion: language agents with verbal reinforcement learning cites WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Gorilla: Large Language Model Connected with Massive APIs
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Reflexion: language agents with verbal reinforcement learning
Builds on17
- Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
- Gorilla: Large Language Model Connected with Massive APIsShishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. GonzalezNeurIPS 2024 · 1,715 citations
- WebShop: Towards Scalable Real-World Web Interaction with Grounded Language AgentsShunyu Yao, Howard Chen, John Yang, Karthik NarasimhanNeurIPS 2022 · 1,477 citations
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual ContextsPan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu et al.ICLR 2024 · 1,472 citations
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu et al.ICLR 2024 · 1,469 citations
Related papers
- Memp: Exploring Agent Procedural MemoryRunnan Fang, Yuan Liang, Xiaobin Wang, Jialong Wu et al.ACL Findings 2026
- Type-Compliant Adaptation CascadesChu-Cheng Lin, Daiyi Peng, Yifeng Lu, Ming Zhang et al.ICLR 2026 · 1 citation
- MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic OptimizationHaidong Xin, Xinze Li, Zhenghao Liu, Yukun Yan et al.ACL Findings 2026
- Structurally Aligned Subtask-Level Memory for Software Engineering AgentsKangning Shen, Jingyuan Zhang, Chenxi Sun, Wencong Zeng et al.ICML 2026 · 9 citations
- Learn Like Humans: Use Meta-cognitive Reflection for Efficient Self-ImprovementXinmeng Hou, Bohao Qu, Wuqi Wang, Peiliang Gong et al.ACL 2026 · 1 citation
