Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model

For the task of fine-grained entity typing (FET), due to the use of a large number of entity types, it is usually considered too costly to manually annotate a training dataset that contains an ample number of examples for each type. A common way to address this problem is to use distantly annotated training examples that contains incorrect labels. But the errors in the automatic annotation may limit the performance of trained models. Recently, there are a few approaches that no longer depend on such weak training data. However, without using sufficient direct entity typing supervision may also cause them to yield inferior performance. In this paper, we propose a new approach that can avoid the need of creating distantly labeled data. We first train an entity typing model that have an extremely broad type coverage by using the ultrafine entity typing data. Then, when there is a need to produce a model for a newly designed fine-grained entity type schema, we can simply fine-tune the previously trained model with a small number of corresponding annotated examples. Experimental results show that our approach achieves outstanding performance for FET under the few-shot setting. It can also outperform state-of-the-art weak supervision based methods after fine-tuning the model with only a small-size manually annotated training set.