EMNLP Findings2022
Generative Prompt Tuning for Relation Classification
Jiale Han, Shuai Zhao, Bo Cheng, Shengkun Ma, Wei Lu
Abstract
Using prompts to explore the knowledge contained within pre-trained language models for downstream tasks has now become an active topic. Current prompt tuning methods mostly convert the downstream tasks to masked language modeling problems by adding clozestyle phrases and mapping all labels to verbalizations with fixed length, which has proven effective for tasks with simple label spaces. However, when applied to relation classification exhibiting complex label spaces, vanilla prompt tuning methods may struggle with label verbalizations with arbitrary lengths due to rigid prompt restrictions. Inspired by the text infilling task for pre-training generative models that can flexibly predict missing spans, we propose a novel generative prompt tuning method to reformulate relation classification as an infilling problem, which frees our approach from limitations of current prompt based approaches and thus fully exploits rich semantics of entity and relation types. In addition, we design entity-guided decoding and discriminative relation scoring to generate and align relations effectively and efficiently during inference. Extensive experiments under fully supervised settings and low-resource settings demonstrate the effectiveness of our approach.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction cites LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
- KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction cites AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
- KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction cites Language Models are Few-Shot Learners
- LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention cites BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts cites Language Models are Few-Shot Learners
- Multilingual Relation Classification via Efficient and Effective Prompting cites BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
- Multilingual Relation Classification via Efficient and Effective Prompting cites KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction
- STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models cites BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
- STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models cites LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
Cited by top-tier papers2
- STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language ModelsMingyu Derek Ma, Xiaoxuan Wang, Po-Nien Kung, P. Jeffrey Brantingham et al.AAAI 2024 · 22 citations
- Multilingual Relation Classification via Efficient and Effective PromptingYuxuan Chen, David Harbecke, Leonhard HennigEMNLP 2022 · 13 citations
Builds on17
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated PromptsTaylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace et al.EMNLP 2020 · 1,162 citations
- LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentionIkuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda et al.EMNLP 2020 · 562 citations
- KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation ExtractionXiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng et al.WWW 2022 · 488 citations
Related papers
- DSP: Discriminative Soft Prompts for Zero-Shot Entity and Relation ExtractionBo Lv, Xin Liu, Shaojie Dai, Nayu Liu et al.ACL Findings 2023 · 11 citations
- Prompt-learning for Fine-grained Entity TypingNing Ding, Yulin Chen, Xu Han, Guangwei Xu et al.EMNLP Findings 2022 · 139 citations
- Enhancing Low-Resource Relation Representations through Multi-View DecouplingChenghao Fan, Wei Wei, Xiaoye Qu, Zhenyi Lu et al.AAAI 2024 · 10 citations
- Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text ClassificationShengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu et al.ACL 2022
- Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation and Instance GenerationJiaxin Huang, Yu Meng, Jiawei HanKDD 2022 · 17 citations
