ACL Findings2025
How Far are LLMs from Being Our Digital Twins? A Benchmark for Persona-Based Behavior Chain Simulation
Rui Li, Heming Xia, Xinfeng Yuan, Qingxiu Dong, Lei Sha, Wenjie Li, Zhifang Sui
Abstract
Recently, LLMs have garnered increasing attention across academic disciplines for their potential as human digital twins, virtual proxies designed to replicate individuals and autonomously perform tasks such as decisionmaking, problem-solving, and reasoning on their behalf. However, current evaluations of LLMs primarily emphasize dialogue simulation while overlooking human behavior simulation, which is crucial for digital twins. To address this gap, we introduce BEHAVIOR-CHAIN, the first benchmark for evaluating LLMs' ability to simulate continuous human behavior. BEHAVIORCHAIN comprises diverse, high-quality, persona-based behavior chains, totaling 15,846 distinct behaviors across 1,001 unique personas, each with detailed history and profile metadata. For evaluation, we integrate persona metadata into LLMs and employ them to iteratively infer contextually appropriate behaviors within dynamic scenarios provided by BEHAVIORCHAIN. Comprehensive evaluation results demonstrated that even state-of-the-art models struggle with accurately simulating continuous human behavior. Resources are available at https://github.com/O- L1RU1/BehaviorChain.
Ask Lune
Ask anything. The answer quotes the paper.
Lune reads all of it and the papers around it, then shows you the passages behind every answer.
Lune answers from
the full paper · 1 top-tier paper that cites it · related work
Free to start. No credit card required.
Cited by top-tier papers1
Builds on10
- Generative Agents: Interactive Simulacra of Human BehaviorJoon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris et al.UIST 2023 · 1,882 citations
- RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language ModelsNoah Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu et al.ACL Findings 2024 · 120 citations
- Character-LLM: A Trainable Agent for Role-PlayingYunfan Shao, Linyang Li, Junqi Dai, Xipeng QiuEMNLP 2023 · 97 citations
- HYDRA: Model Factorization Framework for Black-Box LLM PersonalizationYuchen Zhuang, Haotian Sun, Yue Yu, Rushi Qiang et al.NeurIPS 2024 · 79 citations
- PersonaGym: Evaluating Persona Agents and LLMsVinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari et al.EMNLP Findings 2025 · 11 citations
Related papers
- OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior SimulationZiyi Wang, Yuxuan Lu, Wenbo Li, Amirali Amini et al.ACL 2026 · 26 citations
- TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona SimulationBangde Du, Minghao Guo, Songming He, Ziyi Ye et al.ACL Findings 2026 · 2 citations
- FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial ReasoningZhuohan Xie, Daniil Orel, Rushil Thareja, Dhruv Sahnan et al.ACL 2026 · 13 citations
- Can LLM Agents Simulate Multi-Turn Human Behavior? Evidence from Real Online Customer Behavior DataYuxuan Lu, Jing Huang, Yan Han, Bingsheng Yao et al.ACL 2026 · 17 citations
- Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait AlignmentPriyanka Dey, Aayush Bothra, Yugal Khanter, Jieyu Zhao et al.EMNLP Findings 2025 · 1 citation
