Lune

EMNLP Findings2025

WebCoT: Enhancing Web Agent Reasoning by Reconstructing Chain-of-Thought in Reflection, Branching, and Rollback

Minda Hu, Tianqing Fang, Jianshu Zhang, Jun-Yu Ma, Zhisong Zhang, Jingyan Zhou, Hongming Zhang, Haitao Mi, Dong Yu, Irwin King

Abstract

Web agents powered by Large Language Models (LLMs) show promise for next-generation AI, but their limited reasoning in uncertain, dynamic web environments hinders robust deployment. In this paper, we identify key reasoning skills essential for effective web agents, i.e., reflection & lookahead, branching, and rollback, and curate trajectory data that exemplifies these abilities by reconstructing the agent's (inference-time) reasoning algorithms into chain-of-thought rationales. We conduct experiments in the agent self-improving benchmark, OpenWebVoyager, and demonstrate that distilling salient reasoning patterns into the backbone LLM via simple fine-tuning can substantially enhance its performance. Our approach yields significant improvements across multiple benchmarks, including WebVoyager, Mind2web-live, and SimpleQA (web search), highlighting the potential of targeted reasoning skill enhancement for web agents.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • WebThinker: Empowering Large Reasoning Models with Deep Research Capability cites GAIA: a benchmark for General AI Assistants
  • WebSynthesis: World Model-Guided Monte Carlo Tree Search for Efficient WebAgent Trajectory Synthesis cites WebEvolver: Enhancing Web Agent Self-Improvement with Co-evolving World Model
  • Reflexion: language agents with verbal reinforcement learning cites WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
  • OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis cites Reflexion: language agents with verbal reinforcement learning
  • OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis cites WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
  • WebEvolver: Enhancing Web Agent Self-Improvement with Co-evolving World Model cites WebThinker: Empowering Large Reasoning Models with Deep Research Capability
  • WebEvolver: Enhancing Web Agent Self-Improvement with Co-evolving World Model cites Reflexion: language agents with verbal reinforcement learning
  • WebEvolver: Enhancing Web Agent Self-Improvement with Co-evolving World Model cites GAIA: a benchmark for General AI Assistants
  • WebEvolver: Enhancing Web Agent Self-Improvement with Co-evolving World Model cites WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner CollaborationBowei He, Minda Hu, Zenan Xu, Hongru WANG et al.ICML 2026 · 9 citations
  2. WebSynthesis: World Model-Guided Monte Carlo Tree Search for Efficient WebAgent Trajectory SynthesisYifei Gao, Junhong Ye, Yifan Yang, Jiaqi Wang et al.ACL 2026
  3. WebEvolver: Enhancing Web Agent Self-Improvement with Co-evolving World ModelTianqing Fang, Hongming Zhang, Zhisong Zhang, Kaixin Ma et al.EMNLP 2025

Builds on15

  1. Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
  2. WebShop: Towards Scalable Real-World Web Interaction with Grounded Language AgentsShunyu Yao, Howard Chen, John Yang, Karthik NarasimhanNeurIPS 2022 · 1,477 citations
  3. GAIA: a benchmark for General AI AssistantsGrégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun et al.ICLR 2024 · 716 citations
  4. WebThinker: Empowering Large Reasoning Models with Deep Research CapabilityXiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian et al.NeurIPS 2025 · 354 citations
  5. OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task SynthesisQiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin et al.ACL 2025 · 114 citations

Related papers

  1. Branch-and-Browse: Efficient and Controllable Web Exploration with Tree-Structured Reasoning and Action MemoryShiqi He, Yue Cui, Xinyu Ma, Yaliang Li et al.ACL 2026 · 5 citations
  2. Beneficial Reasoning Behaviors in Agentic Search and Effective Training Methods to Obtain ThemJiahe Jin, Abhijay Sai Paladugu, Chenyan XiongACL Findings 2026
  3. ReSearch: Learning to Reason with Search for LLMs via Reinforcement LearningMingyang Chen, Linzhuang Sun, Tianpeng Li, Haoze Sun et al.NeurIPS 2025 · 125 citations
  4. Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic ToolsJunde Wu, Jiayuan Zhu, Yuyuan Liu, Min Xu et al.ACL 2025 · 88 citations
  5. Large Language Models Can Self-ImproveJiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu et al.EMNLP 2023 · 184 citations
Dusk over the sea between two cliffs drawn in fine vertical lines