ACL Findings2026
LPO: Towards Accurate GUI Agent Interaction via Location Preference Optimization
Jiaqi Tang, Yu Xia, Yi-Feng Wu, Yuwei Hu, Yuhui Chen, Qing-Guo Chen, Xiaogang Xu, Xiangyu Wu, Hao Lu, Yanqing Ma, Shiyin Lu, Qifeng Chen
Abstract
The advent of autonomous agents is transforming interactions with Graphical User Interfaces (GUIs) by employing natural language as a powerful intermediary. Despite the predominance of Supervised Fine-Tuning (SFT) methods in current GUI agents for achieving spatial localization, these methods face substantial challenges due to their limited capacity to accurately perceive positional data. Existing strategies, such as reinforcement learning, often fail to assess positional accuracy effectively, thereby restricting their utility. In response, we introduce Location Preference Optimization (LPO), a novel approach that leverages locational data to optimize interaction preferences. LPO uses information entropy to predict interaction positions by focusing on zones rich in information. Besides, it further introduces a dynamic location reward function based on physical distance, reflecting the varying importance of interaction positions. Supported by Group Relative Preference Optimization (GRPO), LPO facilitates an extensive exploration of GUI environments and significantly enhances interaction precision. Comprehensive experiments demonstrate LPO's superior performance, achieving SOTA results across both offline benchmarks and real-world online evaluations. Our code will be made publicly available soon, at https://github.com/ jqtangust/LPO.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning cites SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
- OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis cites WebLINX: Real-World Website Navigation with Multi-Turn Dialogue
- Learning GUI Grounding with Spatial Reasoning from Visual Feedback cites SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
- GUI-G²: Gaussian Reward Modeling for GUI Grounding cites UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning
- GUI-G²: Gaussian Reward Modeling for GUI Grounding cites SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
- GUI-G²: Gaussian Reward Modeling for GUI Grounding cites OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis
Cited by top-tier papers3
- GUI-G²: Gaussian Reward Modeling for GUI GroundingFei Tang, Zhangxuan Gu, Zhengxi Lu, Xuyang Liu et al.AAAI 2026 · 48 citations
- Learning GUI Grounding with Spatial Reasoning from Visual FeedbackYu Zhao, Wei-Ning Chen, Huseyin Inan, Samuel Kessler et al.ICML 2026 · 9 citations
- Robust-R1: Degradation-Aware Reasoning for Robust Visual UnderstandingJiaqi Tang, Jianmin Chen, Wei Wei, Xiaogang Xu et al.AAAI 2026 · 4 citations
Builds on10
- DAPO: An Open-Source LLM Reinforcement Learning System at ScaleQiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan et al.NeurIPS 2025 · 2,828 citations
- WebLINX: Real-World Website Navigation with Multi-Turn DialogueXing Han Lù, Zdenek Kasner, Siva ReddyICML 2024 · 146 citations
- OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task SynthesisQiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin et al.ACL 2025 · 114 citations
- UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement LearningZhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin et al.AAAI 2026 · 103 citations
- SeeClick: Harnessing GUI Grounding for Advanced Visual GUI AgentsKanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu et al.ACL 2024 · 33 citations
Related papers
- SE-GUI: Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement LearningXinbin Yuan, Jian Jun Zhang, Kaixin Li, Zhuoxuan Cai et al.NeurIPS 2025 · 4 citations
- UI-AGILE: Advancing GUI Agents with Effective Reinforcement Learning and Precise Inference-Time GroundingShuquan Lian, Yuhang Wu, Jia Ma, Yifan Ding, Zihan Song, Bingqi Chen, Xiawu Zheng, Hui Li, Rongrong JiCVPR Findings 2026
- Test-Time Reinforcement Learning for GUI Grounding via Region ConsistencyYong Du, Yuchen Yan, Fei Tang, Zhengxi Lu et al.AAAI 2026 · 10 citations
- InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy OptimizationYuhang Liu, Zeyu Liu, Shuanghe Zhu, Pengxiang Li et al.AAAI 2026 · 15 citations
- GuirlVG: Incentivize GUI Visual Grounding via Empirical Exploration on Reinforcement LearningWeitai Kang, Bin Lei, Gaowen Liu, Caiwen Ding et al.ICLR 2026 · 6 citations
