Simulating Before Planning: Constructing Intrinsic User World Model for User-Tailored Dialogue Policy Planning

Recent advancements in dialogue policy planning have focused on optimizing system agent policies to achieve predefined goals, emphasizing strategy design, trajectory acquisition, and training efficiency. However, these approaches often overlook the critical role of user characteristics, which are essential in real-world scenarios like conversational search and recommendation, where interactions must adapt to individual user traits such as personality, preferences, and goals. To address this gap, we conduct a comprehensive study using task-specific user personas to evaluate dialogue policy planning under diverse user behaviors. Our analysis, based on these user profiles, reveals significant shortcomings in existing approaches, underscoring the necessity for user-tailored dialogue policies. Building on these insights, we propose the User-Tailored Dialogue Policy Planning (UDP) framework, which integrates an Intrinsic User World Model to capture user traits and feedback. UDP operates in three stages: (1) User Persona Portraying, employing a diffusion model to dynamically infer user profiles; (2) User Feedback Anticipating, using a Brownian Bridge-inspired mechanism to predict user reactions; and (3) User-Tailored Policy Planning, synthesizing these elements to optimize response strategies. To enhance robustness, we introduce an active learning approach that prioritizes challenging user personas during training. Extensive experiments across benchmarks, including both collaborative and non-collaborative settings, demonstrate UDP's effectiveness in learning user-specific dialogue strategies. Results confirm the framework's utility, highlighting its robustness, adaptability, and potential to advance user-centric dialogue systems.