OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agents
Bowen Yang, Kaiming Jin, Zhenyu Wu, Zhaoyang Liu, Qiushi Sun, Zehao Li, JingJing Xie, Zhoumianze Liu, Fangzhi Xu, Kanzhi Cheng, Yian Wang, Qingyun Li
Abstract
While Vision-Language Models (VLMs) have significantly advanced Computer-Using Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalization in novel domains. These limitations stem from a lack of granular control over historical visual context curation and the absence of visual-aware tutorial retrieval. To bridge these gaps, we introduce OS-SYMPHONY, a holistic framework that comprises an Orchestrator coordinating two key innovations for robust automation: 1) a Reflection-Memory Agent that utilizes milestone-driven long-term memory to enable trajectory-level self-correction, effectively mitigating visual context loss in longhorizon tasks; 2) Versatile Tool Agents featuring a Multimodal Searcher that adopts a See-Act paradigm to navigate a browser-based sandbox to synthesize live, visually aligned tutorials, thereby resolving fidelity issues in unseen scenarios. Experimental results demonstrate that OS-SYMPHONY delivers substantial performance gains across varying model scales, establishing new state-of-the-art results on three online benchmarks, notably achieving 65.84% on OSWorld. Our code and project are publicly available at OS-Copilot/OS-Symphony and OS-Symphony Homepage.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 95f03bef-750e-4424-ad5a-315700950e6aBuilds on8
- GPT-4V(ision) is a Generalist Web Agent, if GroundedBoyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun et al.ICML 2024 · 496 citations
- WebShaper: Agentically Data Synthesizing via Information-Seeking FormalizationZhengwei Tao, Jialong Wu, Wenbiao Yin, Pu Wu et al.ICLR 2026 · 115 citations
- SeeClick: Harnessing GUI Grounding for Advanced Visual GUI AgentsKanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu et al.ACL 2024 · 33 citations
- OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use AgentsHongrui Jia, Jitong Liao, Xi Zhang, Haiyang Xu et al.ICLR 2026 · 24 citations
- R-WoM: Retrieval-augmented World Model For Computer-use AgentsKai Mei, Jiang Guo, Shuaichen Chang, Mingwen Dong et al.ICLR 2026 · 12 citations
Related papers
- Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video UnderstandingHaiyang Yan, Hongyun Zhou, Peng Xu, Xiaoxue Feng et al.CVPR 2026 · 8 citations
- SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from ExperienceZEYI SUN, Ziyu Liu, Yuhang Zang, Yuhang Cao et al.ICML 2026 · 58 citations
- ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform DataZhaoyang Liu, Jingjing Xie, Zichen Ding, Zehao Li et al.ICLR 2026 · 54 citations
- AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web TutorialsYiheng Xu, Dunjie Lu, Zhennan Shen, Junli Wang et al.ICLR 2025
- GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection BehaviorPenghao Wu, Shengnan Ma, Bo Wang, Jiaheng Yu et al.NeurIPS 2025 · 20 citations
