Graph Hawkes Transformer for Extrapolated Reasoning on Temporal Knowledge Graphs

Temporal Knowledge Graph (TKG) reasoning has attracted increasing attention due to its enormous potential value, and the critical issue is how to model the complex temporal structure information effectively. Recent studies use the method of encoding graph snapshots into hidden vector space and then performing heuristic deductions, which perform well on the task of entity prediction. However, these approaches cannot predict when an event will occur, and have the following limitations: 1) there are many facts not related to the query that can confuse the model; 2) there exists information forgetting caused by long-term evolutionary processes. To this end, we propose a Graph Hawkes Transformer (GHT) for both TKG entity prediction and time prediction tasks in the future time. In GHT, there are two variants of Transformer, which capture the instantaneous structural information and temporal evolution information, respectively, and a new relational continuous-time encoding function to facilitate feature evolution with the Hawkes process. Extensive experiments on four public datasets demonstrate its superior performance, especially on long-term evolutionary tasks. Introduction Knowledge Graph (KG), a multi-relational directed graph database that stores human knowledge and facts, is widely used in downstream applications such as recommendation systems (Guo et al., 2020; Wang et al., 2018) , web search (Paulheim, 2017) and question answering (Saxena et al., 2021) . Conventionally, KGs store each fact in the form of a triplet (𝑠𝑢𝑏 𝑗 𝑒𝑐𝑡, 𝑝𝑟𝑒𝑑𝑖𝑐𝑎𝑡𝑒, 𝑜𝑏 𝑗 𝑒𝑐𝑡). However, many facts may change over time and may contain event-based interaction data. To encode the temporal information, Temporal Knowledge Graph (TKG) is proposed so that * Equal Contribution.