Lune

ACL Findings2023

A Simple Yet Strong Domain-Agnostic De-bias Method for Zero-Shot Sentiment Classification

Yang Zhao, Tetsuya Nasukawa, Masayasu Muraoka, Bishwaranjan Bhattacharjee

Abstract

Zero-shot prompt-based learning has made much progress in sentiment analysis, and considerable effort has been dedicated to designing high-performing prompt templates. However, two problems exist; First, large language models are often biased to their pre-training data, leading to poor performance in prompt templates that models have rarely seen. Second, in order to adapt to different domains, redesigning prompt templates is usually required, which is time-consuming and inefficient. To remedy both shortcomings, we propose a simple yet strong data construction method to debias a given prompt template, yielding a large performance improvement in sentiment analysis tasks across different domains, pre-trained language models, and prompt templates. Also, we demonstrate the advantage of using domainagnostic generic responses over the in-domain ground-truth data. We release the code here 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Solving Aspect Category Sentiment Analysis as a Text Generation Task cites Making Pre-trained Language Models Better Few-shot Learners
  • Calibrate Before Use: Improving Few-shot Performance of Language Models cites Making Pre-trained Language Models Better Few-shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on3

  1. Calibrate Before Use: Improving Few-shot Performance of Language ModelsZihao Zhao, Eric Wallace, Shi Feng, Dan Klein et al.ICML 2021 · 1,843 citations
  2. Solving Aspect Category Sentiment Analysis as a Text Generation TaskJian Liu, Zhiyang Teng, Leyang Cui, Hanmeng Liu et al.EMNLP 2021 · 62 citations
  3. Making Pre-trained Language Models Better Few-shot LearnersTianyu Gao, Adam Fisch, Danqi ChenACL 2021

Related papers

  1. Zero-shot Approach to Overcome Perturbation Sensitivity of PromptsMohna Chakraborty, Adithya Kulkarni, Qi LiACL 2023 · 6 citations
  2. Reducing Spurious Correlations in Aspect-based Sentiment Analysis with Explanation from Large Language ModelsQianlong Wang, Keyang Ding, Bin Liang, Min Yang et al.EMNLP Findings 2023 · 15 citations
  3. Adversarial Soft Prompt Tuning for Cross-Domain Sentiment AnalysisHui Wu, Xiaodong ShiACL 2022 · 98 citations
  4. BayesPrompt: Prompting Large-Scale Pre-Trained Language Models on Few-shot Inference via Debiased Domain AbstractionJiangmeng Li, Fei Song, Yifan Jin, Wenwen Qiang et al.ICLR 2024 · 9 citations
  5. The Benefits of Label-Description Training for Zero-Shot Text ClassificationLingyu Gao, Debanjan Ghosh, Kevin GimpelEMNLP 2023 · 6 citations
Dusk over the sea between two cliffs drawn in fine vertical lines