ACL Findings2026
MemSearcher: Iterative Memory Integration for Search Agent via End-to-End Reinforcement Learning
Qianhao Yuan, Jie Lou, Zichao Li, Jiawei Chen, Yaojie Lu, Hongyu Lin, Le Sun, Debing Zhang, Xianpei Han
Abstract
LLM-based search agents often concatenate the full interaction history into the context, producing long and noisy inputs, and increasing compute cost and GPU memory overhead. To address this issue, we propose Mem-Searcher, an agent framework that maintains a compact memory during multi-turn interactions, retaining only question-relevant information and thereby keeping the context length stable across turns. Training MemSearcher is challenging because each trajectory spans multiple turns under different LLM contexts, making each turn an independent optimization target in reinforcement learning. We introduce multi-context GRPO, which propagates trajectory-level advantages to all turns for end-to-end optimization. Experiments demonstrate that MemSearcher outperforms strong history-concatenation (ReAct-style) baselines on a range of public datasets while maintaining nearly constant token counts across multiturn interactions. The code and models will be publicly available at https://github.com/icip- cas/MemSearcher.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent cites MemoryBank: Enhancing Large Language Models with Long-Term Memory
- HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models cites Measuring and Narrowing the Compositionality Gap in Language Models
- A-Mem: Agentic Memory for LLM Agents cites MemoryBank: Enhancing Large Language Models with Long-Term Memory
Builds on22
- A-Mem: Agentic Memory for LLM AgentsWujiang Xu, Zujie Liang, Kai Mei, Hang Gao et al.NeurIPS 2025 · 1,138 citations
- HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language ModelsBernal Jimenez Gutierrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga et al.NeurIPS 2024 · 395 citations
- MemoryBank: Enhancing Large Language Models with Long-Term MemoryWanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye et al.AAAI 2024 · 394 citations
- Measuring and Narrowing the Compositionality Gap in Language ModelsOfir Press, Muru Zhang, Sewon Min, Ludwig Schmidt et al.EMNLP Findings 2023 · 296 citations
- MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory AgentHongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen et al.ICLR 2026 · 231 citations
Related papers
- MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon AgentsZijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim et al.ICLR 2026 · 223 citations
- EMA: An Episodic Memory Agent for Efficient and Selective MemoryHongyi Lan, Jiaqi Song, Zhengjia Zhong, Hui Li et al.ACL Findings 2026
- Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model AgentsYi Yu, Liuyi Yao, Yuexiang Xie, Qingquan Tan et al.ACL 2026 · 40 citations
- Beyond the Context Window: Scaling Agentic RL via End-to-end Optimized Context CompressionMiao Lu, Weiwei Sun, Weihua Du, Zhan Ling et al.ACL 2026
- Agentic Conversational Search with Contextualized Reasoning via Reinforcement LearningFengran Mo, Yifan Gao, Sha Li, Hansi Zeng et al.ACL Findings 2026
