Lune

ACL Findings2025

Exploring the Choice Behavior of Large Language Models

Weidong Wu, Qinlin Zhao, Hao Chen, Lexin Zhou, Defu Lian, Hong Xie

Abstract

Large Language Models (LLMs) are increasingly deployed as human assistants across various domains where they help to make choices. However, the mechanisms behind LLMs' choice behavior remain unclear, posing risks in safety-critical situations. Inspired by the intrinsic and extrinsic motivation framework within the classic human behavioral model of Self-Determination Theory and its established research methodologies, we investigate the factors influencing LLMs' choice behavior by constructing a virtual QA platform that includes three different experimental conditions, with four models from GPT and Llama series participating in repeated experiments. Our findings indicate that LLMs' behavior is influenced not only by intrinsic attention bias but also by extrinsic social influence, exhibiting patterns similar to the Matthew effect and Conformity. We distinguish independent pathways of these two factors in LLMs' behavior by self-report. This work provides new insights into understanding LLMs' behavioral patterns, exploring their human-like characteristics.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • CompeteAI: Understanding the Competition Dynamics of Large Language Model-based Agents cites Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. Pre-Trained Language Models for Interactive Decision-MakingShuang Li, Xavier Puig, Chris Paxton, Yilun Du et al.NeurIPS 2022 · 341 citations
  2. Automatic Chain of Thought Prompting in Large Language ModelsZhuosheng Zhang, Aston Zhang, Mu Li, Alex SmolaICLR 2023 · 234 citations
  3. CultureLLM: Incorporating Cultural Differences into Large Language ModelsCheng Li, Mengzhuo Chen, Jindong Wang, Sunayana Sitaram et al.NeurIPS 2024 · 101 citations
  4. CompeteAI: Understanding the Competition Dynamics of Large Language Model-based AgentsQinlin Zhao, Jindong Wang, Yixuan Zhang, Yiqiao Jin et al.ICML 2024 · 58 citations
  5. Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology ViewJintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu et al.ACL 2024

Related papers

  1. Conformity in Large Language ModelsXiaochen Zhu, Caiqi Zhang, Tom Stafford, Nigel Collier et al.ACL 2025 · 16 citations
  2. How Personality Traits Shape LLM Risk-Taking BehaviourJohn Hartley, Conor Brian Hamill, Dale Seddon, Devesh Batra et al.ACL Findings 2025 · 2 citations
  3. Large Language Models Assume People are More Rational than We Really areRyan Liu, Jiayi Geng, Joshua C. Peterson, Ilia Sucholutsky et al.ICLR 2025
  4. Will LLMs Sink or Swim? Exploring Decision-Making Under PressureKyusik Kim, Hyeonseok Jeon, Jeongwoo Ryu, Bongwon SuhEMNLP Findings 2024 · 1 citation
  5. Inertia in Moral and Value Judgments of Large Language ModelsBruce W. Lee, Yeongheon Lee, Hyunsoo ChoACL 2026 · 5 citations
Dusk over the sea between two cliffs drawn in fine vertical lines