EMNLP Findings2023
Boosting Prompt-Based Self-Training With Mapping-Free Automatic Verbalizer for Multi-Class Classification
Abstract
Recently, prompt-based fine-tuning has garnered considerable interest as a core technique for few-shot text classification task. This approach reformulates the fine-tuning objective to align with the Masked Language Modeling (MLM) objective. Leveraging unlabeled data, prompt-based self-training has shown greater effectiveness in binary and three-class classification. However, prompt-based self-training for multi-class classification has not been adequately investigated, despite its significant applicability to real-world scenarios. Moreover, extending current methods to multi-class classification suffers from the verbalizer that extracts the predicted value of manually pre-defined single label word for each class from MLM predictions. Consequently, we introduce a novel, efficient verbalizer structure, named Mappingfree Automatic Verbalizer (MAV). Comprising two fully connected layers, MAV serves as a trainable verbalizer that automatically extracts the requisite word features for classification by capitalizing on all available information from MLM predictions. Experimental results on five multi-class classification datasets indicate MAV's superior self-training efficacy. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling cites Unsupervised Data Augmentation for Consistency Training
- FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling cites FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence cites Unsupervised Data Augmentation for Consistency Training
Builds on19
- FixMatch: Simplifying Semi-Supervised Learning with Consistency and ConfidenceKihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang et al.NeurIPS 2020 · 5,129 citations
- Unsupervised Data Augmentation for Consistency TrainingQizhe Xie, Zihang Dai, Eduard H. Hovy, Thang Luong et al.NeurIPS 2020 · 2,774 citations
- SimCSE: Simple Contrastive Learning of Sentence EmbeddingsTianyu Gao, Xingcheng Yao, Danqi ChenEMNLP 2021 · 2,496 citations
- FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo LabelingBowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu et al.NeurIPS 2021 · 1,389 citations
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated PromptsTaylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace et al.EMNLP 2020 · 1,162 citations
Related papers
- SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific TopicsZhiwen You, Kanyao Han, Haotian Zhu, Bertram Ludäscher et al.EMNLP 2024
- Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text ClassificationShengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu et al.ACL 2022
- Towards Unified Prompt Tuning for Few-shot Text ClassificationJianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan et al.EMNLP Findings 2022 · 37 citations
- MetricPrompt: Prompting Model as a Relevance Metric for Few-shot Text ClassificationHongyuan Dong, Weinan Zhang, Wanxiang CheKDD 2023 · 2 citations
- Hierarchical Verbalizer for Few-Shot Hierarchical Text ClassificationKe Ji, Yixin Lian, Jingsheng Gao, Baoyuan WangACL 2023 · 17 citations
