Lune

EMNLP Findings2022

Robust Task-Oriented Dialogue Generation with Contrastive Pre-training and Adversarial Filtering

Shiquan Yang, Xinting Huang, Jey Han Lau, Sarah M. Erfani

Abstract

Data artifacts incentivize machine learning models to learn non-transferable generalizations by taking advantage of shortcuts in the data, and there is growing evidence that data artifacts play a role for the strong results that deep learning models achieve in recent natural language processing benchmarks. In this paper, we focus on task-oriented dialogue and investigate whether popular datasets such as Mul-tiWOZ contain such data artifacts. We found that by only keeping frequent phrases in the training examples, state-of-the-art models perform similarly compared to the variant trained with full data, suggesting they exploit these spurious correlations to solve the task. Motivated by this, we propose a contrastive learning based framework to encourage the model to ignore these cues and focus on learning generalisable patterns. We also experiment with adversarial filtering to remove "easy" training instances so that the model would focus on learning from the "harder" instances. We conduct a number of generalization experiments -e.g., cross-domain/dataset and adversarial tests -to assess the robustness of our approach and found that it works exceptionally well. Source code of our experiments is available at: https://github.com/shiquanyang/ Robust-Dialogue.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Supervised Contrastive Learning cites A Simple Framework for Contrastive Learning of Visual Representations
  • SimCSE: Simple Contrastive Learning of Sentence Embeddings cites A Simple Framework for Contrastive Learning of Visual Representations
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on13

  1. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  2. Supervised Contrastive LearningPrannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna et al.NeurIPS 2020 · 7,049 citations
  3. WinoGrande: An Adversarial Winograd Schema Challenge at ScaleKeisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin ChoiAAAI 2020 · 3,037 citations
  4. SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
  5. Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue DatasetAbhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta et al.AAAI 2020 · 707 citations

Related papers

  1. Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent ClassificationMujeen Sung, James Gung, Elman Mansimov, Nikolaos Pappas et al.EMNLP 2023 · 1 citation
  2. An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot FillingHan Liu, Feng Zhang, Xiaotong Zhang, Siyang Zhao et al.EMNLP Findings 2021 · 20 citations
  3. Group-wise Contrastive Learning for Neural Dialogue GenerationHengyi Cai, Hongshen Chen, Yonghao Song, Zhuoye Ding et al.EMNLP Findings 2020 · 30 citations
  4. CoNT: Contrastive Neural Text GenerationChenxin An, Jiangtao Feng, Kai Lv, Lingpeng Kong et al.NeurIPS 2022 · 37 citations
  5. Debiasing Multimodal Sarcasm Detection with Contrastive LearningMengzhao Jia, Can Xie, Liqiang JingAAAI 2024 · 51 citations
Dusk over the sea between two cliffs drawn in fine vertical lines