Lune

EMNLP Findings2023

TK-KNN: A Balanced Distance-Based Pseudo Labeling Approach for Semi-Supervised Intent Classification

Nicholas Botzer, David Vasquez, Tim Weninger, Issam H. Laradji

Abstract

The ability to detect intent in dialogue systems has become increasingly important in modern technology. These systems often generate a large amount of unlabeled data, and manually labeling this data requires substantial human effort. Semi-supervised methods attempt to remedy this cost by using a model trained on a few labeled examples and then by assigning pseudolabels to further a subset of unlabeled examples that has a model prediction confidence higher than a certain threshold. However, one particularly perilous consequence of these methods is the risk of picking an imbalanced set of examples across classes, which could lead to poor labels. In the present work, we describe Top-K K-Nearest Neighbor (TK-KNN), which uses a more robust pseudo-labeling approach based on distance in the embedding space while maintaining a balanced set of pseudo-labeled examples across classes through a ranking-based approach. Experiments on several datasets show that TK-KNN outperforms existing models, particularly when labeled data is scarce on popular datasets such as CLINC150 and Bank-ing77. Code is available at https://github. com/ServiceNow/tk-knn

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling cites In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning
  • FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling cites FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
  • In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning cites S4L: Self-Supervised Semi-Supervised Learning
  • In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning cites FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
  • FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence cites S4L: Self-Supervised Semi-Supervised Learning
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on15

  1. Supervised Contrastive LearningPrannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna et al.NeurIPS 2020 · 7,049 citations
  2. FixMatch: Simplifying Semi-Supervised Learning with Consistency and ConfidenceKihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang et al.NeurIPS 2020 · 5,129 citations
  3. FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo LabelingBowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu et al.NeurIPS 2021 · 1,389 citations
  4. S4L: Self-Supervised Semi-Supervised LearningLucas Beyer, Xiaohua Zhai, Avital Oliver, Alexander KolesnikovICCV 2019 · 854 citations
  5. In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised LearningMamshad Nayeem Rizve, Kevin Duarte, Yogesh S. Rawat, Mubarak ShahICLR 2021 · 630 citations

Related papers

  1. Discovering New Intents with Deep Aligned ClusteringHanlei Zhang, Hua Xu, Ting-En Lin, Rui LyuAAAI 2021 · 138 citations
  2. New Intent Discovery with Pre-training and Contrastive LearningYuwei Zhang, Haode Zhang, Li-Ming Zhan, Xiao-Ming Wu et al.ACL 2022 · 55 citations
  3. Pseudo-Label Enhanced Prototypical Contrastive Learning for Uniformed Intent DiscoveryYimin Deng, Yuxia Wu, Guoshuai Zhao, Li Zhu et al.EMNLP Findings 2024 · 2 citations
  4. SemPPL: Predicting Pseudo-Labels for Better Contrastive RepresentationsMatko Bosnjak, Pierre Harvey Richemond, Nenad Tomasev, Florian Strub et al.ICLR 2023 · 4 citations
  5. Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent DiscoveryYutao Mou, Keqing He, Pei Wang, Yanan Wu et al.EMNLP 2022 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines