Lune

EMNLP Findings2022

BARLE: Background-Aware Representation Learning for Background Shift Out-of-Distribution Detection

Hanyu Duan, Yi Yang, Ahmed Abbasi, Kar Yan Tam

Abstract

Machine learning models often suffer from a performance drop when they are applied to out-of-distribution (OOD) samples, i.e., those drawn far away from the training data distribution. Existing OOD detection work mostly focuses on identifying semantic-shift OOD samples, e.g., instances from unseen new classes. However, background-shift OOD detection, which identifies samples with domain or style-change, represents a more practical yet challenging task. In this paper, we propose Background-Aware Representation Learning (BARLE) for background-shift OOD detection in NLP. Specifically, we generate semanticspreserving background-shifted pseudo OOD samples from pretrained masked language models. We then contrast the in-distribution (ID) samples with their pseudo OOD counterparts. Unlike prior semantic-shift OOD detection work that often leverages an external text corpus, BARLE only uses ID data, which is more flexible and cost-efficient. In experiments across several text classification tasks, we demonstrate that BARLE is capable of improving background-shift OOD detection performance while maintaining ID classification accuracy. We further investigate the properties of the generated pseudo OOD samples, uncovering the working mechanism of BARLE.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Energy-based Out-of-distribution Detection cites Self-Supervised Learning for Generalizable Out-of-Distribution Detection
  • Contrastive Learning with Hard Negative Samples cites A Simple Framework for Contrastive Learning of Visual Representations
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on21

  1. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  2. Energy-based Out-of-distribution DetectionWeitang Liu, Xiaoyun Wang, John D. Owens, Yixuan LiNeurIPS 2020 · 2,213 citations
  3. Contrastive Learning with Hard Negative SamplesJoshua David Robinson, Ching-Yao Chuang, Suvrit Sra, Stefanie JegelkaICLR 2021 · 999 citations
  4. Self-Supervised Learning for Generalizable Out-of-Distribution DetectionSina Mohseni, Mandar Pitale, J. B. S. Yadawa, Zhangyang WangAAAI 2020 · 229 citations
  5. Selective Question Answering under Domain ShiftAmita Kamath, Robin Jia, Percy LiangACL 2020 · 121 citations

Related papers

  1. Types of Out-of-Distribution Texts and How to Detect ThemUdit Arora, William Huang, He HeEMNLP 2021
  2. Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain DetectionRheeya Uppaal, Junjie Hu, Yixuan LiACL 2023 · 9 citations
  3. Holistic Sentence Embeddings for Better Out-of-Distribution DetectionSishuo Chen, Xiaohan Bi, Rundong Gao, Xu SunEMNLP Findings 2022 · 9 citations
  4. Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained TransformersJaeyoung Kim, Kyuheon Jung, Dongbin Na, Sion Jang et al.ACL Findings 2023 · 6 citations
  5. Rethinking Out-of-distribution (OOD) Detection: Masked Image Modeling is All You NeedJingyao Li, Pengguang Chen, Zexin He, Shaozuo Yu et al.CVPR 2023
Dusk over the sea between two cliffs drawn in fine vertical lines