Learning In-context Learning for Named Entity Recognition

Named entity recognition in real-world applications suffers from the diversity of entity types, the emergence of new entity types, and the lack of high-quality annotations. To address the above problems, this paper proposes an in-context learning-based NER approach, which can effectively inject in-context NER ability into PLMs and recognize entities of novel types on-the-fly using only a few demonstrative instances. Specifically, we model PLMs as a meta-function λ instruction, demonstrations, text .M 1 , and a new entity extractor can be implicitly constructed by applying new instruction and demonstrations to PLMs, i.e., (λ.M)(instruction, demonstrations) → F where F will be a new entity extractor, i.e., F: text → entities. To inject the above in-context NER ability into PLMs, we propose a meta-function pre-training algorithm, which pre-trains PLMs by comparing the (instruction, demonstration)-initialized extractor with a surrogate golden extractor. Experimental results on 4 few-shot NER datasets show that our method can effectively inject in-context NER ability into PLMs and significantly outperforms the PLMs+fine-tuning counterparts. * This work was partially done when Jiawei Chen interned at Baidu. † Corresponding authors. 1 This paper represents functions using lambdacalculus (Barendregt, 1992) , and each function is represented as λx,y,z.M , where x, y, z are variables and M is function definition/abstraction. The function can apply to arguments such as (λ x,y,z .M )(x = A, y = B, z = C) (fully applied) or (λ x,y,z .M )(x = A, y = B) (partially applied). Entities: SARS-CoV-2 is virus. COVID-19 is disease.