Hierarchical Inductive Transfer for Continual Dialogue Learning

Pre-trained models have achieved excellent 001 performance on the dialogue task. However, 002 for the continual increase of online chit-chat 003 scenarios, directly fine-tuning these models for 004 each of the new tasks not only explodes the 005 capacity of the dialogue system on the em-006 bedded devices but also causes knowledge for-007 getting on pre-trained models and knowledge 008 interference between diverse dialogue tasks. 009 In this work, we propose a hierarchical in-010 ductive transfer framework to learn and de-011 ploy the dialogue skills continually and effi-012 ciently. First, we introduce the adapter mod-013 ule into pre-trained models for learning new 014 dialogue tasks. As the only trainable mod-015 ule, it is beneficial for the dialogue system on 016 the embedded devices to acquire new dialogue 017 skills with negligible additional parameters. 018 Then, for alleviating knowledge interference 019 between tasks yet benefiting the regularization 020 between them, we further design hierarchical 021 inductive transfer that enables new tasks to use 022 general knowledge in the base adapter with-023 out being misled by diverse knowledge in task-024 specific adapters. Empirical evaluation and 025 analysis indicate that our framework obtains 026 comparable performance under deployment-027 friendly model capacity.