HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Control
Li Ji, Siyin Wang, Pengfang Qian, Xiaopeng Yu, Yihai Tian, Zhaoye Fei, Jingjing Gong, Xipeng Qiu
Abstract
Current Vision-Language-Action (VLA) models excel at robotic manipulation but often struggle with non-Markovian tasks requiring long-term memory and reasoning due to their reliance on immediate observations. Existing solutions face a "frequency-competence paradox," where stronger reasoning models are too slow for real-time control, while faster models lack sufficient reasoning capabilities. To resolve this architectural misalignment, we propose HiMe, a Hierarchical Embodied Memory framework that decouples embodied intelligence into a high-frequency Executor for execution, a Sentry for working memory, and a Planner for long-term strategy. We also introduce a dynamic knowledge system based on cross-modal semantic schemas and active management mechanisms, allowing robots to maintain memory plasticity through "Add, Update, and Delete" operations. This hierarchical design effectively balances the conflict between real-time execution and slow thinking planning, significantly improving success rates in long-horizon tasks. Experiments demonstrate that this approach not only outperforms flat memory baselines but also exhibits the novel ability to self-correct its internal knowledge based on human preferences.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext d220f831-3e82-42c4-a682-6c1dc34e68a2Builds on13
- A-Mem: Agentic Memory for LLM AgentsWujiang Xu, Zujie Liang, Kai Mei, Hang Gao et al.NeurIPS 2025 · 1,138 citations
- MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic ManipulationHao Shi, Bin Xie, Yingfei Liu, Lin Sun et al.ICLR 2026 · 227 citations
- Describe, Explain, Plan and Select: Interactive Planning with LLMs Enables Open-World Multi-Task AgentsZihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu et al.NeurIPS 2023 · 178 citations
- Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement LearningSikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie et al.ACL 2026 · 140 citations
- Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon TasksZaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen et al.NeurIPS 2024 · 104 citations
Related papers
- RoBridge: A Hierarchical Architecture Bridging Cognition and Execution for General Robotic ManipulationKaidong Zhang, Rongtao Xu, Pengzhen Ren, Junfan Lin et al.ICCV 2025 · 1 citation
- ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent PlanningChi-Pin Huang, Yueh-Hua Wu, Min-Hung Chen, Yu-Chiang Frank Wang et al.NeurIPS 2025 · 179 citations
- Planning with an Embodied Learnable MemoryPriyam Parashar, Jacob Krantz, Matthew Chang, Kavit Shah et al.ICLR 2026
- Scaling up Memory for Robotic Control via Experience RetrievalAjay Sridhar, Jennifer Pan, Satvik Sharma, Chelsea FinnICLR 2026 · 21 citations
- SCOPE: Evolving Symbolic World for Planning in Open-Ended EnvironmentsYundaichuan Zhan, Minghe Gao, Zhongqi Yue, Wendong Bu et al.ICML 2026
