Lune

EMNLP Findings2023

Large Language Models Meet Harry Potter: A Dataset for Aligning Dialogue Agents with Characters

Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, Jia Li

Abstract

In recent years, Dialogue-style Large Language Models (LLMs) such as ChatGPT and GPT4 have demonstrated immense potential in constructing open-domain dialogue agents. However, aligning these agents with specific characters or individuals remains a considerable challenge due to the complexities of character representation and the lack of comprehensive annotations. In this paper, we introduce the Harry Potter Dialogue (HPD) dataset, designed to advance the study of dialogue agents and character alignment. The dataset encompasses all dialogue sessions (in both English and Chinese) from the Harry Potter series and is annotated with vital background information, including dialogue scenes, speakers, character relationships, and attributes. These extensive annotations may empower LLMs to unlock characterdriven dialogue capabilities. Furthermore, it can serve as a universal benchmark for evaluating how well can a LLM aligning with a specific character. We benchmark LLMs on HPD using both fine-tuning and in-context learning settings. Evaluation results reveal that although there is substantial room for improvement in generating high-quality, character-aligned responses, the proposed dataset is valuable in guiding models toward responses that better align with the character of Harry Potter.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

31 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LoRA: Low-Rank Adaptation of Large Language Models cites Language Models are Few-Shot Learners
  • Personalized Generation In Large Model Era: A Survey cites A Pre-Training Based Personalized Dialogue Generation Model with Persona-Sparse Data
  • Enhancing Chat Language Models by Scaling High-quality Instructional Conversations cites LoRA: Low-Rank Adaptation of Large Language Models
  • Enhancing Chat Language Models by Scaling High-quality Instructional Conversations cites Language Models are Few-Shot Learners
  • Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model cites Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment
  • GLM-130B: An Open Bilingual Pre-trained Model cites Language Models are Few-Shot Learners
  • ZeroG: Investigating Cross-dataset Zero-shot Transferability in Graphs cites LoRA: Low-Rank Adaptation of Large Language Models
  • Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment cites Language Models are Few-Shot Learners
  • Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles cites Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
  • Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles cites Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model
  • Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles cites Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers31

  1. Personalized Generation In Large Model Era: A SurveyYiyan Xu, Jinghao Zhang, Alireza Salemi, Xinting Hu et al.ACL 2025 · 45 citations
  2. Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit ProfilesKuang Wang, Xianfei Li, Shenghao Yang, Li Zhou et al.ACL 2025 · 24 citations
  3. ZeroG: Investigating Cross-dataset Zero-shot Transferability in GraphsYuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu et al.KDD 2024 · 19 citations
  4. Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-AlignmentKeming Lu, Bowen Yu, Chang Zhou, Jingren ZhouACL 2024 · 16 citations
  5. Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language ModelYeyong Yu, Runsheng Yu, Haojie Wei, Zhanqiu Zhang et al.ACL 2025 · 13 citations

Builds on10

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  3. GLM-130B: An Open Bilingual Pre-trained ModelAohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang et al.ICLR 2023 · 295 citations
  4. A Pre-Training Based Personalized Dialogue Generation Model with Persona-Sparse DataYinhe Zheng, Rongsheng Zhang, Minlie Huang, Xiaoxi MaoAAAI 2020 · 173 citations
  5. Enhancing Chat Language Models by Scaling High-quality Instructional ConversationsNing Ding, Yulin Chen, Bokai Xu, Yujia Qin et al.EMNLP 2023 · 95 citations

Related papers

  1. CharacterBench: Benchmarking Character Customization of Large Language ModelsJinfeng Zhou, Yongkang Huang, Bosi Wen, Guanqun Bi et al.AAAI 2025 · 7 citations
  2. CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent EvaluationQuan Tu, Shilong Fan, Zihang Tian, Tianhao Shen et al.ACL 2024
  3. Dual Task Framework for Improving Persona-Grounded Dialogue DatasetMinju Kim, Beong-woo Kwak, Youngwook Kim, Hong-in Lee et al.AAAI 2022 · 8 citations
  4. MedDialog: Large-scale Medical Dialogue DatasetsGuangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang et al.EMNLP 2020 · 163 citations
  5. CharacterCraft: Bridging the Literature-Reality Dialogue Gap for Practical Role-Playing AgentsXuyan Yin, Xinran Yang, Zihao Li, Lixin Zou et al.EMNLP Findings 2025
Dusk over the sea between two cliffs drawn in fine vertical lines