Multi-Stage Pre-training Enhanced by ChatGPT for Multi-Scenario Multi-Domain Dialogue Summarization

Dialogue summarization involves a wide range of scenarios and domains.However, existing methods generally only apply to specific scenarios or domains.In this study, we propose a new pre-trained model specifically designed for multi-scenario multi-domain dialogue summarization.It adopts a multi-stage pre-training strategy to reduce the gap between the pretraining objective and fine-tuning objective.Specifically, we first conduct domain-aware pre-training using large-scale multi-scenario multi-domain dialogue data to enhance the adaptability of our pre-trained model.Then, we conduct task-oriented pre-training using largescale multi-scenario multi-domain "dialoguesummary" parallel data annotated by ChatGPT to enhance the dialogue summarization ability of our pre-trained model.Experimental results on three dialogue summarization datasets from different scenarios and domains indicate that our pre-trained model significantly outperforms previous state-of-the-art models in full fine-tuning, zero-shot, and few-shot settings 1 .Empathetic Dialogues Decode Wizard of Wikipedia Wizard of Internet Topical-Chat Persona-Chat Blended Skill Talk ConvAI2 MuTual CMU DoG DailyDialog DialogRE FriendsQA DREAM 14 open-domain dialogue datasets across multiple scenarios 6 multi-domain customer service dialogue datasets MetaLWOz Schema Taskmaster2 MultiWOZ 2.