ACL Findings2022
Hierarchical Inductive Transfer for Continual Dialogue Learning
Shaoxiong Feng, Xuancheng Ren, Kan Li, Xu Sun
Abstract
Pre-trained models have achieved excellent performance on the dialogue task. However, for the continual increase of online chit-chat scenarios, directly fine-tuning these models for each of the new tasks not only explodes the capacity of the dialogue system on the embedded devices but also causes knowledge forgetting on pre-trained models and knowledge interference among diverse dialogue tasks. In this work, we propose a hierarchical inductive transfer framework to learn and deploy the dialogue skills continually and efficiently. First, we introduce the adapter module into pre-trained models for learning new dialogue tasks. As the only trainable module, it is beneficial for the dialogue system on the embedded devices to acquire new dialogue skills with negligible additional parameters. Then, for alleviating knowledge interference between tasks yet benefiting the regularization between them, we further design hierarchical inductive transfer that enables new tasks to use general knowledge in the base adapter without being misled by diverse knowledge in task-specific adapters. Empirical evaluation and analysis indicate that our framework obtains comparable performance under deployment-friendly model capacity.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Generating Relevant and Coherent Dialogue Responses using Self-Separated Conditional Variational AutoEncoders cites Regularizing Dialogue Generation by Imitating Implicit Scenarios
- Multi-View Feature Representation for Dialogue Generation with Bidirectional Distillation cites Regularizing Dialogue Generation by Imitating Implicit Scenarios
Cited by top-tier papers2
Builds on5
- Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend SkillsEric Michael Smith, Mary Williamson, Kurt Shuster, Jason Weston et al.ACL 2020 · 18 citations
- Regularizing Dialogue Generation by Imitating Implicit ScenariosShaoxiong Feng, Xuancheng Ren, Hongshen Chen, Bin Sun et al.EMNLP 2020 · 18 citations
- Multi-View Feature Representation for Dialogue Generation with Bidirectional DistillationShaoxiong Feng, Xuancheng Ren, Kan Li, Xu SunAAAI 2021 · 13 citations
- What to Pre-Train on? Efficient Intermediate Task SelectionClifton Poth, Jonas Pfeiffer, Andreas Rücklé, Iryna GurevychEMNLP 2021 · 8 citations
- Generating Relevant and Coherent Dialogue Responses using Self-Separated Conditional Variational AutoEncodersBin Sun, Shaoxiong Feng, Yiwei Li, Jiamou Liu et al.ACL 2021
Related papers
- TaSL: Continual Dialog State Tracking via Task Skill Localization and ConsolidationYujie Feng, Xu Chu, Yongxin Xu, Guangyuan Shi et al.ACL 2024 · 1 citation
- Continual Prompt Tuning for Dialog State TrackingQi Zhu, Bing Li, Fei Mi, Xiaoyan Zhu et al.ACL 2022
- Continual Dialogue State Tracking via Example-Guided Question AnsweringHyundong Cho, Andrea Madotto, Zhaojiang Lin, Khyathi Raghavi Chandu et al.EMNLP 2023
- FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain DialogueAlon Albalak, Yi-Lin Tuan, Pegah Jandaghi, Connor Pryor et al.EMNLP 2022 · 2 citations
- Continual Learning in Task-Oriented Dialogue SystemsAndrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon et al.EMNLP 2021 · 68 citations
