Lune

UbiComp2026顶会

Act2Intention: A Benchmark For Developing Active Mobile Agents Through Inferring User Intention from GUI Actions

Xiaokai Yan, Jingtao Ding, Yong Li, Zhiwen Yu

摘要

Mobile GUI Agents powered by multimodal large language models (MLLMs) show promise in human-computer intelligence. However, current research primarily focuses on reactive task execution while lacking a comprehensive "understandingprediction-execution" process for user intentions, which are the core requirements of active agents. In this paper, we propose the Act2Intention framework that builds an active mobile agent by integrating understanding, predicting user intentions, and executing decisions. Firstly, we constructed the Act2Intention Bench through collection and validated generation, comprising 72,511 intentions and over 700,000 actions across 52 apps, thereby establishing the first benchmark for evaluating proactive agents via continuous "intention-actions" trajectories. We further develop the Act2Intention Agent, achieving proactive services through Proactive-oriented Intention Understanding, Personalized Proactive Intention Prediction, and Experienceguided Intention Execution. Experimental results show that supervised fine-tuning on Act2Intention Bench yields absolute improvements of +32.0 Acc-S, +10.25 Acc-S, and +6.9 SSR points over non-fine-tuned counterparts under the same agent framework for intention understanding, prediction, and execution, respectively. This success underscores the necessity and value of the Act2Intention Bench, which establishes a standardized platform for developing and evaluating proactive agents and consequently paves the way for research on intention-driven human-computer interaction. To facilitate further research, we open-source our code and Act2Intention Bench at: npuNancy/Act2Intention.

问 Lune

几分钟读透这篇 Paper,不用再花一下午。

Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。

可以从这些问题问起

Lune 可以查阅

全文 · 相关的 Paper

获取答案

免费开始,无需绑卡

脉络

  • Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration 引用了 AutoDroid: LLM-powered Task Automation in Android
  • UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning 引用了 Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration
它引用的引用它的从左到右按年份排列,引用越多,位置越高。

它引用的 Paper29

  1. Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent CollaborationJunyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang 等NeurIPS 2024 · 245 次引用
  2. ProAgent: Building Proactive Cooperative Agents with Large Language ModelsCeyao Zhang, Kaijie Yang, Siyi Hu, Zihao Wang 等AAAI 2024 · 141 次引用
  3. OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task SynthesisQiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin 等ACL 2025 · 114 次引用
  4. UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement LearningZhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin 等AAAI 2026 · 103 次引用
  5. AutoDroid: LLM-powered Task Automation in AndroidHao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao 等MobiCom 2024 · 94 次引用

相关 Paper

  1. FingerTip 20K: A Benchmark for Proactive and Personalized Mobile LLM AgentsQinglong Yang, Haoming Li, Haotian Zhao, Xiaokai Yan 等ICLR 2026 · 21 次引用
  2. ProactiveMobile: A Comprehensive Benchmark for Boosting Proactive Intelligence On Mobile DevicesDezhi Kong, Zhengzhao Feng, Qiliang Liang, Hao Wang 等CVPR 2026 · 6 次引用
  3. Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile AgentsShihan Deng, Weikai Xu, Hongda Sun, Wei Liu 等ACL 2024 · 10 次引用
  4. Spa-Bench: a comprehensive Benchmark for Smartphone Agent EvaluationJingxuan Chen, Derek Yuen, Bin Xie, Yuhao Yang 等ICLR 2025
  5. ContextAgent: Context-Aware Proactive LLM Agents with Open-world Sensory PerceptionsBufang Yang, Lilin Xu, Liekang Zeng, Kaiwei Liu 等NeurIPS 2025 · 68 次引用
黄昏的海面,两侧是细线勾勒的悬崖