Anchor: Branch-Point Data Generation for GUI Agents
Jinbiao Wei, Yilun Zhao, Kangqi Ni, Arman Cohan
Abstract
End-to-end GUI agents for real desktop environments require large amounts of high-quality interaction data, yet collecting human demonstrations is expensive and existing synthetic pipelines often suffer from limited task diversity or noisy, goal-drifting trajectories. We present a trajectory expansion framework Anchor that bootstraps scalable desktop supervision from a small set of verified seed demonstrations. Starting from each seed, we identify branch points that correspond to meaningful state changes and propose new, state-grounded task variants conditioned on the current GUI context. An executing agent then follows the proposed instructions to generate new trajectories, while a verifier enforces task completion via state-aware checks and trajectory-level consistency. To improve supervision quality, we further apply task-conditioned step-level filtering to remove ungrounded actions and denoise post-branch segments to maintain coherent intent. Experiments on standard desktop benchmarks, OSWorld and WindowsAgentArena, show that models fine-tuned on our expanded corpus achieve consistent improvements over zero-shot agents and representative synthesis baselines, and generalize across applications and operating systems.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 0edbf644-f276-4a0a-a4cc-95b4fed27196Cited by top-tier papers1
Ask how each one uses itBuilds on5
- GPT-4V(ision) is a Generalist Web Agent, if GroundedBoyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun et al.ICML 2024 · 496 citations
- WebLINX: Real-World Website Navigation with Multi-Turn DialogueXing Han Lù, Zdenek Kasner, Siva ReddyICML 2024 · 146 citations
- Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple ActionsGuo Gan, Yuxuan Ding, Cong Chen, Yuwei Ren et al.ACL 2026 · 6 citations
- Agent S: An Open Agentic Framework that Uses Computers Like a HumanSaaket Agashe, Jiuzhou Han, Shuyu Gan, Jiachen Yang et al.ICLR 2025 · 2 citations
- AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web TutorialsYiheng Xu, Dunjie Lu, Zhennan Shen, Junli Wang et al.ICLR 2025
Related papers
- Watch and Learn: Learning to Use Computers from Online VideosChan Hee Song, Yiwen Song, Palash Goyal, Yu Su et al.CVPR 2026 · 8 citations
- Grounding Computer Use Agents on Human DemonstrationsAarash Feizi, Shravan Nayak, Xiangru Jian, Kevin Qinghong Lin et al.ICLR 2026 · 16 citations
- TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI AgentsBofei Zhang, Zirui Shang, Zhi Gao, Wang Zhang et al.AAAI 2026 · 26 citations
- UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and InteractionShravan Nayak, Xiangru Jian, Kevin Qinghong Lin, Juan A. Rodríguez et al.ICML 2025
- Efficient Agent Training for Computer UseYanheng He, Jiahe Jin, Pengfei LiuICLR 2026 · 15 citations
