Lune

ACL Findings2025

How Far are LLMs from Being Our Digital Twins? A Benchmark for Persona-Based Behavior Chain Simulation

Rui Li, Heming Xia, Xinfeng Yuan, Qingxiu Dong, Lei Sha, Wenjie Li, Zhifang Sui

摘要

Recently, LLMs have garnered increasing attention across academic disciplines for their potential as human digital twins, virtual proxies designed to replicate individuals and autonomously perform tasks such as decisionmaking, problem-solving, and reasoning on their behalf. However, current evaluations of LLMs primarily emphasize dialogue simulation while overlooking human behavior simulation, which is crucial for digital twins. To address this gap, we introduce BEHAVIOR-CHAIN, the first benchmark for evaluating LLMs' ability to simulate continuous human behavior. BEHAVIORCHAIN comprises diverse, high-quality, persona-based behavior chains, totaling 15,846 distinct behaviors across 1,001 unique personas, each with detailed history and profile metadata. For evaluation, we integrate persona metadata into LLMs and employ them to iteratively infer contextually appropriate behaviors within dynamic scenarios provided by BEHAVIORCHAIN. Comprehensive evaluation results demonstrated that even state-of-the-art models struggle with accurately simulating continuous human behavior. Resources are available at https://github.com/O- L1RU1/BehaviorChain.

问 Lune

尽管问,回答会引用原文。

Lune 会读完全文和相关的 Paper,每个回答都附上所依据的原文段落。

可以从这些问题问起

Lune 的依据

全文 · 引用它的 1 篇顶会 Paper · 相关的 Paper

获取答案

免费开始,无需绑卡

引用它的顶会 Paper1

  1. HalluCitation Matters: Revealing the Impact of Hallucinated References with 300 Hallucinated Papers in ACL ConferencesYusuke Sakai, Hidetaka Kamigaito, Taro WatanabeACL 2026 · 18 次引用

它引用的 Paper10

  1. Generative Agents: Interactive Simulacra of Human BehaviorJoon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris 等UIST 2023 · 1,882 次引用
  2. RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language ModelsNoah Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu 等ACL Findings 2024 · 120 次引用
  3. Character-LLM: A Trainable Agent for Role-PlayingYunfan Shao, Linyang Li, Junqi Dai, Xipeng QiuEMNLP 2023 · 97 次引用
  4. HYDRA: Model Factorization Framework for Black-Box LLM PersonalizationYuchen Zhuang, Haotian Sun, Yue Yu, Rushi Qiang 等NeurIPS 2024 · 79 次引用
  5. PersonaGym: Evaluating Persona Agents and LLMsVinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari 等EMNLP Findings 2025 · 11 次引用

相关 Paper

  1. OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior SimulationZiyi Wang, Yuxuan Lu, Wenbo Li, Amirali Amini 等ACL 2026 · 26 次引用
  2. TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona SimulationBangde Du, Minghao Guo, Songming He, Ziyi Ye 等ACL Findings 2026 · 2 次引用
  3. FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial ReasoningZhuohan Xie, Daniil Orel, Rushil Thareja, Dhruv Sahnan 等ACL 2026 · 13 次引用
  4. Can LLM Agents Simulate Multi-Turn Human Behavior? Evidence from Real Online Customer Behavior DataYuxuan Lu, Jing Huang, Yan Han, Bingsheng Yao 等ACL 2026 · 17 次引用
  5. Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait AlignmentPriyanka Dey, Aayush Bothra, Yugal Khanter, Jieyu Zhao 等EMNLP Findings 2025 · 1 次引用
黄昏的海面,两侧是细线勾勒的悬崖