Lune

EMNLP Findings2025

Chain of Strategy Optimization Makes Large Language Models Better Emotional Supporter

Weixiang Zhao, Xingyu Sui, Xinyang Han, Yang Deng, Yulin Hu, Jiahe Guo, Libo Qin, Qianyun Du, Shijin Wang, Yanyan Zhao, Bing Qin, Ting Liu

Abstract

The growing emotional stress in modern society has increased the demand for Emotional Support Conversations (ESC). While Large Language Models (LLMs) show promise for ESC, they face two key challenges: (1) low strategy selection accuracy, and (2) preference bias, limiting their adaptability to users' emotional needs. Existing supervised finetuning (SFT) struggles to address these issues, as it rigidly trains models on single goldstandard responses without modeling nuanced strategy trade-offs. To overcome these limitations, we propose a novel two-stage framework that optimizes strategy selection preferences at each dialogue turn. We first leverage Monte Carlo Tree Search to construct ESC-Pro, a high-quality preference dataset with turnlevel strategy-response pairs. Then training on ESC-Pro with Chain-of-Strategy Optimization (CSO) improves both strategy accuracy and bias mitigation, enabling LLMs to generate more empathetic and contextually appropriate responses. Experiments on LLaMA-3.1-8B, Gemma-2-9B, and Qwen2.5-7B demonstrate that CSO outperforms standard SFT, highlighting the efficacy of fine-grained, turn-level preference modeling in ESC. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LoRA: Low-Rank Adaptation of Large Language Models cites Language Models are Few-Shot Learners
  • Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Language Models are Few-Shot Learners
  • SimPO: Simple Preference Optimization with a Reference-Free Reward cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
  • SimPO: Simple Preference Optimization with a Reference-Free Reward cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic AgentsPeisong Wang, Ruotian Ma, Bang Zhang, Xingyu Chen et al.ICLR 2026 · 28 citations
  2. Simulated Rewards, Skewed Strategies: Tracing the Acquired Preference Bias in LLM-Based Dialogue PlannersHeyan Huang, Yizhe Yang, Huashan Sun, Jiawei Li et al.AAAI 2026

Builds on16

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  3. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  4. Self-Refine: Iterative Refinement with Self-FeedbackAman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan et al.NeurIPS 2023 · 4,972 citations
  5. SimPO: Simple Preference Optimization with a Reference-Free RewardYu Meng, Mengzhou Xia, Danqi ChenNeurIPS 2024 · 1,203 citations

Related papers

  1. Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support ConversationDongjin Kang, Sunghwan Kim, Taeyoon Kwon, Seungjun Moon et al.ACL 2024 · 14 citations
  2. DecoupledESC: Enhancing Emotional Support Generation via Strategy-Response Decoupled Preference OptimizationChao Zhang, Xin Shi, Xueqiao Zhang, Yifan Zhu et al.EMNLP Findings 2025 · 3 citations
  3. SoulChat: Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy ConversationsYirong Chen, Xiaofen Xing, Jingkai Lin, Huimin Zheng et al.EMNLP Findings 2023 · 81 citations
  4. ESCA: An Emotional Support Conversation Agent for Enhancing Reasonable Strategy Planning and Effective ExpressionJing Li, Yanxin Luo, Donghong Han, Yimeng Zhan et al.AAAI 2026
  5. From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference AlignmentJing Ye, Lu Xiang, Yaping Zhang, Chengqing ZongEMNLP Findings 2025 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines