Lune

EMNLP Findings2024

BASES: Large-scale Web Search User Simulation with Large Language Model based Agents

Ruiyang Ren, Peng Qiu, Yingqi Qu, Jing Liu, Xin Zhao, Hua Wu, Ji-Rong Wen, Haifeng Wang

Abstract

Due to the excellent capacities of large language models (LLMs), it becomes feasible to develop LLM-based agents for reliable user simulation. Considering the scarcity and limit (e.g., privacy issues) of real user data, in this paper, we conduct large-scale user simulations for the web search scenario to improve the analysis and modeling of user search behavior. Specially, we propose BASES, a novel user simulation framework with LLM-based agents, designed to facilitate comprehensive simulations of web search user behaviors. Our simulation framework can generate unique user profiles at scale, which subsequently leads to diverse search behaviors. To demonstrate the effectiveness of BASES, we conduct evaluation experiments based on two human benchmarks in both Chinese and English, demonstrating that BASES can effectively simulate large-scale human-like search behaviors. To further accommodate the research on web search, we develop WARRIORS, a new large-scale dataset encompassing web search user behaviors, including both Chinese and English versions, which can greatly bolster research in the field of information retrieval.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

6 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TOME: A Two-stage Approach for Model-based Retrieval cites RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking
  • Crucible: Quantifying the Potential of Control Algorithms through LLM Agents cites Generative Agents: Interactive Simulacra of Human Behavior
  • Crucible: Quantifying the Potential of Control Algorithms through LLM Agents cites Beyond Interpretability: Exploring the Comprehensibility of Adaptive Video Streaming through Large Language Models
  • Beyond Interpretability: Exploring the Comprehensibility of Adaptive Video Streaming through Large Language Models cites Generative Agents: Interactive Simulacra of Human Behavior
  • Self-Calibrated Listwise Reranking with Large Language Models cites TOME: A Two-stage Approach for Model-based Retrieval
  • Self-Calibrated Listwise Reranking with Large Language Models cites RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking
  • REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering cites RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking
  • ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate cites Generative Agents: Interactive Simulacra of Human Behavior
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers6

  1. Self-Calibrated Listwise Reranking with Large Language ModelsRuiyang Ren, Yuhao Wang, Kun Zhou, Wayne Xin Zhao et al.WWW 2025 · 12 citations
  2. REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question AnsweringYuhao Wang, Ruiyang Ren, Junyi Li, Xin Zhao et al.EMNLP 2024 · 12 citations
  3. Policy Learning from Tutorial Books via Understanding, Rehearsing and IntrospectingXiong-Hui Chen, Ziyan Wang, Yali Du, Shengyi Jiang et al.NeurIPS 2024 · 4 citations
  4. Crucible: Quantifying the Potential of Control Algorithms through LLM AgentsLianchen Jia, Chaoyang Li, Qian Houde, Tianchi Huang et al.NeurIPS 2025 · 1 citation
  5. Beyond Interpretability: Exploring the Comprehensibility of Adaptive Video Streaming through Large Language ModelsLianchen Jia, Chaoyang Li, Ziqi Yuan, Jiahui Chen et al.ACM MM 2025 · 1 citation

Builds on7

  1. Generative Agents: Interactive Simulacra of Human BehaviorJoon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris et al.UIST 2023 · 1,882 citations
  2. ChatEval: Towards Better LLM-based Evaluators through Multi-Agent DebateChi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu et al.ICLR 2024 · 871 citations
  3. RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingRuiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao et al.EMNLP 2021 · 147 citations
  4. Group based Personalized Search by Integrating Search Behaviour and Friend NetworkYujia Zhou, Zhicheng Dou, Bingzheng Wei, Ruobing Xie et al.SIGIR 2021 · 29 citations
  5. TOME: A Two-stage Approach for Model-based RetrievalRuiyang Ren, Wayne Xin Zhao, Jing Liu, Hua Wu et al.ACL 2023 · 14 citations

Related papers

  1. A Computational Framework for Evaluating Human-likeness in LLMs' Open-ended Human BehaviorsYuxuan Lei, Jianxun Lian, Defu Lian, Jincenzi Wu et al.ICML 2026
  2. ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping AssistantsPei Wang, Yanan Wu, Xiaoshuai Song, Weixun Wang et al.ACL 2026 · 5 citations
  3. LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory FeedbackThai Quoc Hoang, Kung-Hsiang Huang, Shirley Kokane, Jianguo Zhang et al.ACL Findings 2025 · 1 citation
  4. Large Language Models Empowered Personalized Web AgentsHongru Cai, Yongqi Li, Wenjie Wang, Fengbin Zhu et al.WWW 2025 · 62 citations
  5. WideSearch: Benchmarking Agentic Broad Info-SeekingRyan Wong, Jiawei Wang, Junjie Zhao, Li Chen et al.ICLR 2026 · 66 citations
Dusk over the sea between two cliffs drawn in fine vertical lines