Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding

We propose an entity-agnostic representation learning method for handling the problem of inefficient parameter storage costs brought by embedding knowledge graphs. Conventional knowledge graph embedding methods map elements in a knowledge graph, including entities and relations, into continuous vector spaces by assigning them one or multiple specific embeddings (i.e., vector representations). Thus the number of embedding parameters increases linearly as the growth of knowledge graphs. In our proposed model, Entity-Agnostic Representation Learning (EARL), we only learn the embeddings for a small set of entities and refer to them as reserved entities. To obtain the embeddings for the full set of entities, we encode their distinguishable information from their connected relations, k-nearest reserved entities, and multi-hop neighbors. We learn universal and entityagnostic encoders for transforming distinguishable information into entity embeddings. This approach allows our proposed EARL to have a static, efficient, and lower parameter count than conventional knowledge graph embedding methods. Experimental results show that EARL uses fewer parameters and performs better on link prediction tasks than baselines, reflecting its parameter efficiency. Recently, many knowledge graphs (KGs) (Pan et al. 2017), including Freebase (Bollacker et al. 2008), NELL (Carlson et al. 2010), Wikidata (Vrandečić and Krötzsch 2014), and YAGO (Tanon, Weikum, and Suchanek 2020) have been used as the knowledge resource for a myriad of applications in the field of natural language processing (Xiong et al. 2020; Yu et al. 2022), as well as in the study of computer vision (Huang et al. 2020; Chen et al. 2021b). Typically, knowledge graphs contain a large number of factual triples in the form of (head entity, relation, tail entity), or (h, r, t) for short. A triple reflects a specific connection (i.e., relation) between two entities / concepts.