Belief Updating and Delegation in Multi-Task Human-AI Interaction: Evidence from Controlled Simulations
Shreyan Biswas, Alexander Erlei, Ujwal Gadiraju
摘要
Large language models (LLMs) increasingly support heterogeneous tasks within a single interface, requiring users to form, update, and act upon beliefs about one system across domains with different reliability profiles. Understanding how such beliefs transfer across tasks and shape delegation is therefore critical for the design of multipurpose AI systems. We report a preregistered experiment (N = 240, 7,200 trials) in which participants interacted with a controlled AI simulation across grammar checking, travel planning, and visual question answering, each with fixed, domain-typical accuracy levels. Delegation was operationalized as a binary reliance decision—accepting the AI’s output versus acting independently and belief dynamics were evaluated against Bayesian benchmarks. We find three main results. First, participants do not reset beliefs between tasks: priors in a new task depend on posteriors from the previous task, with a 10-point increase predicting a 3–4 point higher subsequent prior. Second, within tasks, belief updating follows the Bayesian direction but is substantially conservative, proceeding at roughly half the normative Bayesian rate. Third, delegation is driven primarily by subjective beliefs about AI accuracy rather than self-confidence, though confidence independently reduces reliance when beliefs are held constant. Together, these findings show that users form global, path-dependent expectations about multipurpose AI systems, update them conservatively, and rely on AI primarily based on subjective beliefs rather than objective performance. We discuss implications for expectation calibration, reliance design, and the risks of belief spillovers in deployed LLM-based interfaces.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper2
- When Life Gives You AI, Will You Turn It Into A Market for Lemons? Understanding How Information Asymmetries About AI System Capabilities Affect Market Outcomes and AdoptionAlexander Erlei, Federico Maria Cau, Radoslav Georgiev, Sagar Chethan Kumar 等CHI 2026 · 被引用 3 次
- The Data-Dollars Tradeoff: Privacy Harms vs. Economic Risk in Personalized AI AdoptionAlexander Erlei, Tahir Abbas, Kilian Bizer, Ujwal GadirajuCHI 2026 · 被引用 1 次
它引用的顶会 Paper23
- The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge WorkersHao-Ping (Hank) Lee, Advait Sarkar, Lev Tankelevitch, Ian Drosos 等CHI 2025 · 被引用 690 次
- TravelPlanner: A Benchmark for Real-World Planning with Language AgentsJian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu 等ICML 2024 · 被引用 376 次
- CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model CapabilitiesMina Lee, Percy Liang, Qian YangCHI 2022 · 被引用 340 次
- Is the Most Accurate AI the Best Teammate? Optimizing AI for TeamworkGagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz 等AAAI 2021 · 被引用 185 次
- Conceptual Metaphors Impact Perceptions of Human-AI CollaborationPranav Khadpe, Ranjay Krishna, Li Fei-Fei, Jeffrey T. Hancock 等CSCW 2020 · 被引用 179 次
相关 Paper
- To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language ModelsJessica Y. Bo, Sophia Wan, Ashton AndersonCHI 2025 · 被引用 31 次
- Understanding Choice Independence and Error Types in Human-AI CollaborationAlexander Erlei, Abhinav Sharma, Ujwal GadirajuCHI 2024 · 被引用 25 次
- Behavioral Indicators of Overreliance During Interaction with Conversational Language ModelsChang Liu, Qinyi Zhou, Xinjie Shen, Xingyu Bruce Liu 等CHI 2026 · 被引用 4 次
- Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different LanguagesShreyan Biswas, Alexander Erlei, Ujwal GadirajuCHI 2025 · 被引用 11 次
- The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use AgentsWeihao Xuan, Qingcheng Zeng, Heli Qi, Yunze Xiao 等ACL 2026 · 被引用 4 次
