Lune

NeurIPS2025Top-tier venue

Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning

Marwa Abdulhai, Ryan Cheng, Donovan Clay, Tim Althoff, Sergey Levine, Natasha Jaques

Abstract

Large Language Models (LLMs) are increasingly used to simulate human users in interactive settings such as therapy, education, and social role-play. While these simulations enable scalable training and evaluation of AI agents, off-the-shelf LLMs often drift from their assigned personas, contradict earlier statements, or abandon role-appropriate behavior. We introduce a unified framework for evaluating and improving persona consistency in LLM-generated dialogue. We define three automatic metrics-prompt-to-line consistency, line-to-line consistency, and Q&A consistency-that capture different types of persona drift and validate each against human annotations. Using these metrics as reward signals, we apply multiturn reinforcement learning to fine-tune LLMs for three user roles: a patient, a student, and a social chat partner. Our method reduces inconsistency by over 55%, resulting in more coherent, faithful, and trustworthy simulated users.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization cites Training language models to follow instructions with human feedback
  • Synthia: Scalable Grounded Persona Generation from Social Media Data cites Generative Agents: Interactive Simulacra of Human Behavior
  • AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback cites Generative Agents: Interactive Simulacra of Human Behavior
  • AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback cites Training language models to follow instructions with human feedback
  • CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society cites Training language models to follow instructions with human feedback
  • Generative Agents: Interactive Simulacra of Human Behavior cites Training language models to follow instructions with human feedback
  • HumanLM: Simulating Users with State Alignment Beats Response Imitation cites Generative Agents: Interactive Simulacra of Human Behavior
  • HumanLM: Simulating Users with State Alignment Beats Response Imitation cites Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
  • Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies cites Training language models to follow instructions with human feedback
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Simulated Students in Tutoring Dialogues: Substance or Illusion?Alexander Scarlatos, Jaewook Lee, Simon Woodhead, Andrew LanACL 2026 · 6 citations
  2. Synthia: Scalable Grounded Persona Generation from Social Media DataVahid Rahimzadeh, Erfan Moosavi Monazzah, Mohammad Taher Pilehvar, Yadollah YaghoobzadehACL 2026 · 1 citation
  3. Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic OptimizationYihang Yao, Zhepeng Cen, Haohong Lin, Shiqi Liu et al.ICML 2026
  4. HumanLM: Simulating Users with State Alignment Beats Response ImitationShirley Wu, Evelyn Choi, Arpandeep Khatua, Zhanghan Wang et al.ICML 2026

Builds on24

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model SocietyGuohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin et al.NeurIPS 2023 · 1,975 citations
  3. Generative Agents: Interactive Simulacra of Human BehaviorJoon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris et al.UIST 2023 · 1,882 citations
  4. AlpacaFarm: A Simulation Framework for Methods that Learn from Human FeedbackYann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang et al.NeurIPS 2023 · 948 citations
  5. Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject StudiesGati V. Aher, Rosa I. Arriaga, Adam Tauman KalaiICML 2023 · 651 citations

Related papers

  1. Are Economists Always More Introverted? Analyzing Consistency in Persona-Assigned LLMsManon Reusens, Bart Baesens, David JurgensEMNLP Findings 2025 · 1 citation
  2. Can LLM Agents Maintain a Persona in Discourse?Pranav Bhandari, Nicolas Fay, Michael J. Wise, Amitava Datta et al.EMNLP 2025
  3. Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended GenerationJisu Shin, Juhyun Oh, Eunsu Kim, Hoyun Song et al.ACL Findings 2025 · 1 citation
  4. The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMsPengrui Han, Rafal Kocielnik, Peiyang Song, Ramit Debnath et al.ICML 2026 · 33 citations
  5. LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language ModelsMarwa Abdulhai, Isadora White, Charlie Victor Snell, Charles Sun et al.ICML 2025
Dusk over the sea between two cliffs drawn in fine vertical lines