Advancing Multimodal Agent Reasoning with Long-Term Neuro-Symbolic Memory
Rongjie Jiang, Jianwei Wang, Gengda Zhao, Chengyang Luo, Kai Wang, Wenjie Zhang
Abstract
Recent advances in large language models have driven the emergence of intelligent agents operating in open-world, multimodal environments. To support long-term reasoning, such agents are typically equipped with external memory systems. However, most existing multimodal agent memories rely primarily on neural representations and vector-based retrieval, which are well-suited for inductive, intuitive reasoning but fundamentally limited in supporting analytical, deductive reasoning critical for real-world decision making. To address this limitation, we propose NS-Mem, a long-term neuro-symbolic memory framework designed to advance multimodal agent reasoning by integrating neural memory with explicit symbolic structures and rules. Specifically, NS-Mem is organized around three core components of a memory system: (1) a three-layer memory architecture that consists of an episodic layer, a semantic layer, and a logic layer, (2) a memory construction and maintenance mechanism implemented by SK-Gen that automatically consolidates structured knowledge from accumulated multimodal experiences and incrementally updates both neural representations and symbolic rules, and (3) a hybrid memory retrieval mechanism that combines similarity-based search with deterministic symbolic query functions to support structured reasoning. Experiments on real-world multimodal reasoning benchmarks demonstrate that Neuro-Symbolic Memory achieves an average 4.35 percentage points improvement in overall reasoning accuracy over pure neural memory systems, with gains of up to 12.5 percentage points on constrained reasoning queries, validating the effectiveness of NS-Mem.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 533f7857-2b1e-4967-bf04-366e99c79eadBuilds on15
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
- ViperGPT: Visual Inference via Python Execution for ReasoningDídac Surís, Sachit Menon, Carl VondrickICCV 2023 · 732 citations
- PAL: Program-aided Language ModelsLuyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon et al.ICML 2023 · 700 citations
- On the Planning Abilities of Large Language Models - A Critical InvestigationKarthik Valmeekam, Matthew Marquez, Sarath Sreedharan, Subbarao KambhampatiNeurIPS 2023 · 509 citations
- MemoryBank: Enhancing Large Language Models with Long-Term MemoryWanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye et al.AAAI 2024 · 394 citations
Related papers
- Symbolic Working Memory Enhances Language Models for Complex Rule ApplicationSiyuan Wang, Zhongyu Wei, Yejin Choi, Xiang RenEMNLP 2024 · 6 citations
- NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks in Open DomainsWonje Choi, Jinwoo Park, Sanghyun Ahn, Daehee Lee et al.ICLR 2025
- Leibniz: Theory-of-Mind Driven Neuro-Symbolic Logical Reasoning via Multi-Agent CollaborationYue Fan, Hu Zhang, Yunxiao Zhao, Guangjun Zhang et al.ACL 2026
- SymAgent: A Neural-Symbolic Self-Learning Agent Framework for Complex Reasoning over Knowledge GraphsBen Liu, Jihai Zhang, Fangquan Lin, Cheng Yang et al.WWW 2025 · 21 citations
- Neuro-Sym Supporter: A Thoughtful Emotion Support Agent Integrating Neural and Symbolic Policy LearningMinghui Ma, Bin Guo, Mengqi Chen, Jingqi Liu et al.WWW 2026
