Co-EPG: A Framework for Co-Evolution of Planning and Grounding in Autonomous GUI Agents
Yuan Zhao, Hualei Zhu, Tingyu Jiang, Shen Li, Xiaohang Xu, Hao Henry Wang
摘要
Graphical User Interface (GUI) task automation constitutes a critical frontier in artificial intelligence research. While effective GUI agents synergistically integrate planning and grounding capabilities, current methodologies exhibit two fundamental limitations: (1) insufficient exploitation of cross-model synergies, and (2) over-reliance on synthetic data generation without sufficient utilization. To address these challenges, we propose Co-EPG, a self-iterative training framework for Co-Evolution of Planning and Grounding. Co-EPG establishes an iterative positive feedback loop: through this loop, the planning model explores superior strategies under grounding-based reward guidance via Group Relative Policy Optimization (GRPO), generating diverse data to optimize the grounding model. Concurrently, the optimized Grounding model provides more effective rewards for subsequent GRPO training of the planning model, fostering continuous improvement. Co-EPG thus enables iterative enhancement of agent capabilities through self-play optimization and training data distillation. On the Multimodal-Mind2Web and AndroidControl benchmarks, our framework outperforms existing state-of-the-art methods after just three iterations without requiring external data. The agent consistently improves with each iteration, demonstrating robust self-enhancement capabilities. This work establishes a novel training paradigm for GUI agents, shifting from isolated optimization to an integrated, self-driven co-evolution approach.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper16
- Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 被引用 3,729 次
- FlashAttention-2: Faster Attention with Better Parallelism and Work PartitioningTri DaoICLR 2024 · 被引用 2,600 次
- ZeRO: memory optimizations toward training trillion parameter modelsSamyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong HeSC 2020 · 被引用 852 次
- A Real-World WebAgent with Planning, Long Context Understanding, and Program SynthesisIzzeddin Gur, Hiroki Furuta, Austin V. Huang, Mustafa Safdari 等ICLR 2024 · 被引用 359 次
- Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility GenerationJiawei Wang, Renhe Jiang, Chuang Yang, Zengqing Wu 等NeurIPS 2024 · 被引用 181 次
相关 Paper
- MobileIPL: Enhancing Mobile Agents Thinking Process via Iterative Preference LearningHuang Kun, Weikai Xu, Yuxuan Liu, Quandong Wang 等ICLR 2026 · 被引用 9 次
- GUI0: Self-Evolving Foundational GUI Agents in Super App EcosystemsXinyi Wang, Wei Dai, Kyle Qiao, Ke Wang 等ACL 2026
- GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection BehaviorPenghao Wu, Shengnan Ma, Bo Wang, Jiaheng Yu 等NeurIPS 2025 · 被引用 20 次
- Measure Twice, Click Once: Co-evolving Proposer and Visual Critic via Reinforcement Learning for GUI GroundingWenkai Wang, Xiyun Li, Hongcan Guo, Wenhao Yu 等ACL 2026 · 被引用 1 次
- GUI-Rise: Structured Reasoning and History Summarization for GUI NavigationTao Liu, Chongyu Wang, Rongjie Li, Yingchen Yu 等NeurIPS 2025 · 被引用 4 次
