Lune

ACL Findings2026

RoleCDE: Benchmarking and Mitigating Role-Alignment Trade-offs in Role-Playing Agents

Huayi Lai, Shichao Song, Simin Niu, Hanyu Wang, Jiawei Yang, Zhouxing Wang, Zhiqiang Yin, Xun Liang

摘要

Role-playing agents(RPAs) are widely used to steer large language models(LLMs) toward role-consistent behavior, yet existing benchmarks mainly evaluate surface-level fidelity and offer limited insight into decision making under role-alignment value conflicts. To address this gap, we introduce RoleCDE, the first benchmark designed to evaluate RPAs under structured conflicts between role-specific values and alignment-oriented constraints. RoleCDE formulates role-aware decision making as cognitive dilemma scenarios, jointly evaluating role-scenario grounding, value conflict resolution, and decision tendencies. The benchmark is constructed at scale, covering approximately 8k diverse role profiles and scenarios and nearly 24k dilemma instances across three difficulty levels and eight role categories. Evaluation of several mainstream LLMs reveals a "Role Value Decoupling" phenomenon, where agents systematically default to alignment-and morality-consistent decisions rather than rolespecific values when the two conflict, even under explicit role conditioning. This behavior is largely invariant to dilemma difficulty but varies substantially across role categories. We further show that RoleCDE-based fine-tuning effectively mitigates this decoupling by improving value trade-off reasoning, while preserving general role-playing fidelity and general reasoning performance. Code is available at: https: //github.com/rabbitrose/RoleCDE.

问 Lune

尽管问,回答会引用原文。

Lune 会读完全文和相关的 Paper,每个回答都附上所依据的原文段落。

可以从这些问题问起

Lune 的依据

全文 · 相关的 Paper

获取答案

免费开始,无需绑卡

它引用的 Paper14

  1. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger 等ICLR 2020 · 8,443 次引用
  2. Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou 等ICLR 2021 · 7,905 次引用
  3. TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 次引用
  4. Generative Agents: Interactive Simulacra of Human BehaviorJoon Sung Park, Joseph C. O'Brien, Carrie Jun Cai, Meredith Ringel Morris 等UIST 2023 · 1,882 次引用
  5. Efficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng 等SOSP 2023 · 1,016 次引用

相关 Paper

  1. RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-FollowingJunru Lu, Jiazheng Li, Guodong Shen, Lin Gui 等ACL Findings 2025
  2. RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs' Contextual SensitivityJisu Shin, Hoyun Song, Juhyun Oh, Changgeon Ko 等ACL Findings 2026
  3. Too Good to be Bad: On the Failure of LLMs to Role-Play VillainsZihao Yi, Qingxuan Jiang, Ruotian Ma, Xingyu Chen 等ACL Findings 2026
  4. SimBench: Benchmarking the Ability of Large Language Models to Simulate Human BehaviorsTiancheng Hu, Joachim Baumann, Lorenzo Lupo, Nigel Collier 等ICLR 2026 · 61 次引用
  5. Pressure Reveals Character: Behavioural Alignment Evaluation at DepthNora Petrova, John BurdenICML 2026
黄昏的海面,两侧是细线勾勒的悬崖