ACL Findings2026
Learning How to Remember: A Meta-Cognitive Management Method for Structured and Transferable Agent Memory
Sirui Liang, Pengfei Cao, Jian Zhao, Wenhao Teng, Xiangwen Liao, Jun Zhao, Kang Liu
Abstract
Large language model (LLM) agents increasingly rely on accumulated memory to solve long-horizon decision-making tasks. However, most existing approaches store memory in fixed representations and reuse it at a single or implicit level of abstraction, which limits generalization and often leads to negative transfer when distribution shift. This paper proposes the Meta-Cognitive Memory Abstraction method (MCMA), which treats memory abstraction as a learnable cognitive skill rather than a fixed design choice. MCMA decouples task execution from memory management by combining a frozen task model with a learned memory copilot. The memory copilot is trained using direct preference optimization, it determines how memories should be structured, abstracted, and reused. Memories are further organized into a hierarchy of abstraction levels, enabling selective reuse based on task similarity. When no memory is transferable, MCMA transfers the ability to abstract and manage memory by transferring the memory copilot. Experiments demonstrate substantial improvements in performance, outof-distribution generalization, and cross-task transfer over several baselines. Our code is available at MCMA.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Reflexion: language agents with verbal reinforcement learning cites ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
- ExpeL: LLM Agents Are Experiential Learners cites ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
- ExpeL: LLM Agents Are Experiential Learners cites Reflexion: language agents with verbal reinforcement learning
Builds on23
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
- ALFWorld: Aligning Text and Embodied Environments for Interactive LearningMohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk et al.ICLR 2021 · 819 citations
- ExpeL: LLM Agents Are Experiential LearnersAndrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin et al.AAAI 2024 · 484 citations
- Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer ControlLongtao Zheng, Rundong Wang, Xinrun Wang, Bo AnICLR 2024 · 132 citations
Related papers
- Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model AgentsYi Yu, Liuyi Yao, Yuexiang Xie, Qingquan Tan et al.ACL 2026 · 40 citations
- From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over MemoryYishuo Cai, Xingyu Guo, Xuancheng Huang, Jinhua Du et al.ICML 2026 · 1 citation
- MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic OptimizationHaidong Xin, Xinze Li, Zhenghao Liu, Yukun Yan et al.ACL Findings 2026
- Memp: Exploring Agent Procedural MemoryRunnan Fang, Yuan Liang, Xiaobin Wang, Jialong Wu et al.ACL Findings 2026
- Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement LearningSikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie et al.ACL 2026 · 140 citations
