Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model

The rapid advancement of large language models (LLMs) has revolutionized role-playing, enabling the development of general role-playing models. However, current role-playing training has two significant issues: (I) Using a predefined role profile to prompt dialogue training for specific scenarios usually leads to biases and even conflicts between the dialogue and the profile, resulting in training biases. (II) Models learn to imitate the role based solely on the profile, neglecting profile-dialogue alignment at the sentence level. To overcome the aforementioned hurdles, we propose a novel framework BEYOND DIALOGUE, which introduces "beyond dialogue" tasks to align dialogue with profile traits for each scenario, eliminating biases during training. Furthermore, the framework achieves a sentence-level fine-grained alignment between profile and dialogue through an innovative prompting mechanism that generates reasoning data for training. Moreover, the aforementioned methods are fully automated and low-cost. Experimental results demonstrate our model excels in adhering to role profiles, outperforming most proprietary general and specialized role-playing baselines. The code and data are provided in https:// github.com/yuyouyu32/BeyondDialogue . 1. Split the novel by tokens & Filter chunks by roles frequency "Look at this," said Ron … "Light?" said Harry…Snape had already taken Harry's invisibility … "It's a trap," Ron said suddenly… "No," said Harry, trying to sound confident. "I think we'll be all right." 2. Extract scenes & Evaluate chunks using role expressiveness LLMs chunk-1: … chunk-2: … chunk-3: … chunk-1 chunk-2 scene-1: Quirrell … scene-2: Tension … scene: Harry struggles …. ① Discard non-single scene chunk chunk-3 chunk-2 ② Keep chunks with role profile reflection Score: 9.0. Score: 3.0. Harry's speech is plain… Harry is courageous in exploration … 5. Align profile and dialogue & Generate derivative beyond dialogue data Personality Analyze the MBTI personality reflected by the characters in the dialogue.