Advancing Multimodal Agent Reasoning with Long-Term Neuro-Symbolic Memory
Rongjie Jiang, Jianwei Wang, Gengda Zhao, Chengyang Luo, Kai Wang, Wenjie Zhang
摘要
Recent advances in large language models have driven the emergence of intelligent agents operating in open-world, multimodal environments. To support long-term reasoning, such agents are typically equipped with external memory systems. However, most existing multimodal agent memories rely primarily on neural representations and vector-based retrieval, which are well-suited for inductive, intuitive reasoning but fundamentally limited in supporting analytical, deductive reasoning critical for real-world decision making. To address this limitation, we propose NS-Mem, a long-term neuro-symbolic memory framework designed to advance multimodal agent reasoning by integrating neural memory with explicit symbolic structures and rules. Specifically, NS-Mem is organized around three core components of a memory system: (1) a three-layer memory architecture that consists of an episodic layer, a semantic layer, and a logic layer, (2) a memory construction and maintenance mechanism implemented by SK-Gen that automatically consolidates structured knowledge from accumulated multimodal experiences and incrementally updates both neural representations and symbolic rules, and (3) a hybrid memory retrieval mechanism that combines similarity-based search with deterministic symbolic query functions to support structured reasoning. Experiments on real-world multimodal reasoning benchmarks demonstrate that Neuro-Symbolic Memory achieves an average 4.35 percentage points improvement in overall reasoning accuracy over pure neural memory systems, with gains of up to 12.5 percentage points on constrained reasoning queries, validating the effectiveness of NS-Mem.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper15
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni 等NeurIPS 2020 · 被引用 19,162 次
- ViperGPT: Visual Inference via Python Execution for ReasoningDídac Surís, Sachit Menon, Carl VondrickICCV 2023 · 被引用 732 次
- PAL: Program-aided Language ModelsLuyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon 等ICML 2023 · 被引用 700 次
- On the Planning Abilities of Large Language Models - A Critical InvestigationKarthik Valmeekam, Matthew Marquez, Sarath Sreedharan, Subbarao KambhampatiNeurIPS 2023 · 被引用 509 次
- MemoryBank: Enhancing Large Language Models with Long-Term MemoryWanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye 等AAAI 2024 · 被引用 394 次
相关 Paper
- Symbolic Working Memory Enhances Language Models for Complex Rule ApplicationSiyuan Wang, Zhongyu Wei, Yejin Choi, Xiang RenEMNLP 2024 · 被引用 6 次
- NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks in Open DomainsWonje Choi, Jinwoo Park, Sanghyun Ahn, Daehee Lee 等ICLR 2025
- Leibniz: Theory-of-Mind Driven Neuro-Symbolic Logical Reasoning via Multi-Agent CollaborationYue Fan, Hu Zhang, Yunxiao Zhao, Guangjun Zhang 等ACL 2026
- SymAgent: A Neural-Symbolic Self-Learning Agent Framework for Complex Reasoning over Knowledge GraphsBen Liu, Jihai Zhang, Fangquan Lin, Cheng Yang 等WWW 2025 · 被引用 21 次
- Neuro-Sym Supporter: A Thoughtful Emotion Support Agent Integrating Neural and Symbolic Policy LearningMinghui Ma, Bin Guo, Mengqi Chen, Jingqi Liu 等WWW 2026
