ACL Findings2025
MotiveBench: How Far Are We From Human-Like Motivational Reasoning in Large Language Models?
Xixian Yong, Jianxun Lian, Xiaoyuan Yi, Xiao Zhou, Xing Xie
摘要
Large language models (LLMs) have been widely adopted as the core of agent frameworks in various scenarios, such as social simulations and AI companions. However, the extent to which they can replicate human-like motivations remains an underexplored question. Existing benchmarks are constrained by simplistic scenarios and the absence of character identities, resulting in an information asymmetry with real-world situations. To address this gap, we propose MOTIVEBENCH, which consists of 200 rich contextual scenarios and 600 reasoning tasks covering multiple levels of motivation. Using MOTIVEBENCH, we conduct extensive experiments on seven popular model families, comparing different scales and versions within each family. The results show that even the most advanced LLMs still fall short in achieving human-like motivational reasoning. Our analysis reveals key findings, including the difficulty LLMs face in reasoning about "love & belonging" motivations and their tendency toward excessive rationality and idealism. These insights highlight a promising direction for future research on the humanization of LLMs. The dataset, benchmark, and code are available at https://aka.ms/motivebench.
问 Lune
尽管问,回答会引用原文。
Lune 会读完全文和相关的 Paper,每个回答都附上所依据的原文段落。
引用它的顶会 Paper3
- Think or Not? Exploring Thinking Efficiency in Large Reasoning Models via an Information-Theoretic LensXixian Yong, Xiao Zhou, Yingying Zhang, Jinlin Li 等NeurIPS 2025 · 44 次引用
- Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language ModelsHanze Guo, Jing Yao, Xiao Zhou, Xiaoyuan Yi 等NeurIPS 2025 · 6 次引用
- Inflated Excellence or True Performance? Rethinking Medical Diagnostic Benchmarks with Dynamic EvaluationXiangxu Zhang, Lei Li, Yanyun Zhou, Xiao Zhou 等ACL 2026 · 3 次引用
它引用的 Paper11
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma 等NeurIPS 2022 · 22,562 次引用
- Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject StudiesGati V. Aher, Rosa I. Arriaga, Adam Tauman KalaiICML 2023 · 651 次引用
- Large Language Models Are Not Robust Multiple Choice SelectorsChujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou 等ICLR 2024 · 424 次引用
- Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic EncodersYupeng Hou, Jiacheng Li, Xiangjun Fu, Zhankui He 等ACL 2026 · 346 次引用
- SOTOPIA: Interactive Evaluation for Social Intelligence in Language AgentsXuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang 等ICLR 2024 · 288 次引用
相关 Paper
- ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language ModelsYuanyi Ren, Haoran Ye, Hanjun Fang, Xin Zhang 等ACL 2024
- SocialBench: Sociality Evaluation of Role-Playing Conversational AgentsHongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu 等ACL Findings 2024 · 16 次引用
- AgentBench: Evaluating LLMs as AgentsXiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu 等ICLR 2024 · 748 次引用
- SimBench: Benchmarking the Ability of Large Language Models to Simulate Human BehaviorsTiancheng Hu, Joachim Baumann, Lorenzo Lupo, Nigel Collier 等ICLR 2026 · 61 次引用
- Read the Room: Video Social Reasoning with Mental-Physical Causal ChainsLixing Niu, Jiapeng Li, Xingping Yu, Xinyi Dong 等ICLR 2026
