From Schema to State: Zero-Shot Scheme-Only Dialogue State Tracking via Diverse Synthetic Dialogue and Step-by-Step Distillation
Huan Xu, Zequn Li, Wen Tang, Jian Jun Zhang
Abstract
Dialogue State Tracking (DST) is crucial for linking user intentions to appropriate services in task-oriented dialogue systems. We propose a zero-shot, scheme-only approach that tackles two main challenges: generating synthetic dialogues that balance diversity with schema alignment, and efficiently distilling knowledge from a large language model (LLM) into a smaller model. Our pipeline first creates scenarios, dialogue logic flows, and utterances via dynamic complexity prompting, eliminating reliance on handcrafted templates. We then use a twostage distillation process to learn formalized dialogue representations and DST related chainof-thought reasoning. This structure preserves interpretive capabilities while reducing inference overhead. Experiments on the MultiWOZ benchmark show that our method achieves state-of-the-art performance under zero-shot, scheme-only situations and generalizes to fewshot scenarios effectively, offering a practical and scalable solution for domains that lack real data. Our code is available 1
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 6afd6ea9-aea8-4568-bf64-ad5a029f8b8cCited by top-tier papers1
Ask how each one uses itBuilds on11
- Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Promptbreeder: Self-Referential Self-Improvement via Prompt EvolutionChrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero et al.ICML 2024 · 432 citations
- Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language ModelsLei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu et al.ACL 2023 · 249 citations
- Dialogue State Tracking with a Language Model using Schema-Driven PromptingChia-Hsuan Lee, Hao Cheng, Mari OstendorfEMNLP 2021 · 87 citations
Related papers
- DiSTRICT: Dialogue State Tracking with Retriever Driven In-Context TuningPraveen Venkateswaran, Evelyn Duesterwald, Vatche IsahagianEMNLP 2023 · 7 citations
- A Dual Prompt Learning Framework for Few-Shot Dialogue State TrackingYuting Yang, Wenqiang Lei, Pei Huang, Juan Cao et al.WWW 2023 · 20 citations
- Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State TrackingQingyue Wang, Liang Ding, Yanan Cao, Yibing Zhan et al.ACL 2023 · 5 citations
- Zero-Shot Dialogue State Tracking via Cross-Task TransferZhaojiang Lin, Bing Liu, Andrea Madotto, Seungwhan Moon et al.EMNLP 2021
- Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State TrackingGiovanni Campagna, Agata Foryciarz, Mehrad Moradshahi, Monica S. LamACL 2020 · 5 citations
