EMNLP Findings2022
Controllable Dialogue Simulation with In-context Learning
Zekun Li, Wenhu Chen, Shiyang Li, Hong Wang, Jing Qian, Xifeng Yan
Abstract
Building dialogue systems requires a large corpus of annotated dialogues.Such datasets are usually created via crowdsourcing, which is expensive and time-consuming.In this paper, we propose DIALOGIC 1 , a novel dialogue simulation method based on large language model in-context learning to automate dataset creation.Seeded with a few annotated dialogues, DIA-LOGIC automatically selects in-context examples for demonstration and prompts GPT-3 to generate new dialogues and annotations in a controllable way.Our method can rapidly expand a small set of dialogue data with minimum or zero human involvement and parameter update and is thus much more cost-efficient and time-saving than crowdsourcing.Experimental results on the MultiWOZ dataset demonstrate that training a model on the simulated dialogues leads to even better performance than using the same amount of human-generated dialogues under the challenging low-resource settings, with as few as 85 dialogues as a seed.When enough data is available, our method can still serve as an effective data augmentation method.Human evaluation results also show that our simulated dialogues have near-human fluency and annotation accuracy.The code and data are available at https://github.com/Leezekun/dialogic. User([hotel]type is guest house, area is center): I need a place to stay.It should be a guest house in the center of town . GPT-3 GenerationTurn-level Belief State Dialog-level Belief State Dialogue Act <hotel, type, guest house> <hotel, area, center> <hotel, request, day> <hotel, request, people> <hotel, type, guest house> <hotel, area, center> <hotel, book stay, 2> <hotel, book people, 3> <hotel, book day, Friday> <hotel, type, guest house> <hotel, area, center> <hotel, book stay, 2> <hotel, book people, 3> <hotel, book day, Friday> <hotel, offerbooked, reference> <general, reqmore, none> System([hotel] [request] day people): how many days and people will be staying ?System([hotel] [offerbooked] reference [general] [reqmore]): the booking was successful.You reference number is [value_reference] .Can I help you with anything else ?User([hotel] stay is 2 , people is 3, day is Friday): we have 3 people, and we plan to stay 2 nights starting from Friday .
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
10 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society cites A Simple Language Model for Task-Oriented Dialogue
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society cites Language Models are Few-Shot Learners
- TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue cites Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue Dataset
- Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System cites A Simple Language Model for Task-Oriented Dialogue
- Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System cites Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue Dataset
- Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System cites Language Models are Few-Shot Learners
- Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System cites TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue
Cited by top-tier papers10
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model SocietyGuohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin et al.NeurIPS 2023 · 1,975 citations
- Lift Yourself Up: Retrieval-augmented Text Generation with Self-MemoryXin Cheng, Di Luo, Xiuying Chen, Lemao Liu et al.NeurIPS 2023 · 177 citations
- Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat DataCanwen Xu, Daya Guo, Nan Duan, Julian J. McAuleyEMNLP 2023 · 112 citations
- SODA: Million-scale Dialogue Distillation with Social Commonsense ContextualizationHyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West et al.EMNLP 2023 · 60 citations
- MEDSAGE: Enhancing Robustness of Medical Dialogue Summarization to ASR Errors with LLM-generated Synthetic DialoguesKuluhan Binici, Abhinav Ramesh Kashyap, Viktor Schlegel, Andy T. Liu et al.AAAI 2025 · 10 citations
Builds on12
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue DatasetAbhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta et al.AAAI 2020 · 707 citations
- A Simple Language Model for Task-Oriented DialogueEhsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz et al.NeurIPS 2020 · 590 citations
- Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue SystemYixuan Su, Lei Shu, Elman Mansimov, Arshit Gupta et al.ACL 2022 · 218 citations
- TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented DialogueChien-Sheng Wu, Steven C. H. Hoi, Richard Socher, Caiming XiongEMNLP 2020 · 210 citations
Related papers
- Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from InstructionsBiswesh Mohapatra, Gaurav Pandey, Danish Contractor, Sachindra JoshiEMNLP Findings 2021 · 20 citations
- q2d: Turning Questions into Dialogs to Teach Models How to SearchYonatan Bitton, Shlomi Cohen-Ganor, Ido Hakimi, Yoad Lewenberg et al.EMNLP 2023 · 3 citations
- AugESC: Dialogue Augmentation with Large Language Models for Emotional Support ConversationChujie Zheng, Sahand Sabour, Jiaxin Wen, Zheng Zhang et al.ACL Findings 2023 · 49 citations
- Paraphrase Augmented Task-Oriented Dialog GenerationSilin Gao, Yichi Zhang, Zhijian Ou, Zhou YuACL 2020 · 78 citations
- Enhancing Dialogue State Tracking Models through LLM-backed User-Agents SimulationCheng Niu, Xingguang Wang, Xuxin Cheng, Juntong Song et al.ACL 2024
