DSP: Discriminative Soft Prompts for Zero-Shot Entity and Relation Extraction

Prompt-based methods have shown their efficacy in transferring general knowledge within pre-trained language models (PLMs) for lowresource scenarios. Typically, prompt-based methods convert downstream tasks to clozestyle problems and map all labels to verbalizers. However, when applied to zero-shot entity and relation extraction, vanilla promptbased methods may struggle with the limited coverage of verbalizers to labels and the slow inference speed. In this work, we propose a novel Discriminative Soft Prompts (DSP) approach to take advantage of the prompt-based methods to strengthen the transmission of general knowledge. Specifically, we develop a discriminative prompt method, which reformulates zero-shot tasks into token discrimination tasks without having to construct verbalizers. Furthermore, to improve the inference speed of the prompt-based methods, we design a soft prompt co-reference strategy, which leverages soft prompts to approximately refer to the vector representation of text tokens. The experimental results demonstrate that, our model outperforms baselines on two zero-shot entity recognition datasets with higher inference speed, and obtains a 7.5% average relation F1score improvement over previous state-of-theart models on Wiki-ZSL and FewRel.