Continual Dialogue State Tracking via Example-Guided Question Answering

Dialogue systems are frequently updated to accommodate new services (e.g. booking restaurants, setting alarm clocks, etc.), but naive updates with new data compromises performance on previous services due to catastrophic forgetting. To mitigate this issue, we propose a simple but powerful reformulation for dialogue state tracking (DST), a key component of dialogue systems that estimates the user's goal as a conversation proceeds. We restructure DST to eliminate service-specific structured text and unify data from all services by decomposing each DST sample to a bundle of fine-grained example-guided question answering tasks. Our reformulation encourages a model to learn the general skill of learning from an in-context example to correctly answer a natural language question that corresponds to a slot in a dialogue state. With a retriever trained to find examples that introduce similar updates to dialogue states, we find that our method can significantly boost continual learning performance, even for a model with just 60M parameters. When combined with dialogue-level memory replay, our approach attains state-of-the-art performance on continual learning metrics without relying on any complex regularization or parameter expansion methods.