Hybrid Self-evolving Structured Memory for Computer-Use Agents

The remarkable progress of vision-language models (VLMs) has enabled computer-use agents to interact with computers in a humanlike manner. Yet real-world computer-use tasks remain difficult due to long-horizon workflows, diverse interfaces, and frequent intermediate errors. Prior work equips agents with external memory built from large collections of trajectories, but relies on flat retrieval over discrete summaries or continuous embeddings, falling short of the structured organization and selfevolving characteristics of human memory. Inspired by the brain, we propose Hybrid Selfevolving Structured Memory (HyMEM), a graph-based memory that couples discrete highlevel symbolic nodes with continuous trajectory embeddings. HyMEM maintains a graph structure to support multi-hop retrieval, selfevolution via node update operations, and onthe-fly working-memory refreshing during inference. Extensive experiments show that HyMEM consistently improves open-source computer-use agents, enabling 7B/8B backbones to match or surpass strong closed-source models; notably, it boosts Qwen2.5-VL-7B by +22.5% and outperforms Gemini2.5-Pro-Vision and GPT-4o.