Lune

ACL Findings2021

Incorporating Global Information in Local Attention for Knowledge Representation Learning

Yu Zhao, Han Zhou, Ruobing Xie, Fuzhen Zhuang, Qing Li, Ji Liu

Abstract

Graph Attention Networks (GATs) have proven a promising model that takes advantage of localized attention mechanism to perform knowledge representation learning (KRL) on graph-structure data, e.g., Knowledge Graphs (KGs). While such approaches model entities' local pairwise importance, they lack the capability to model global importance relative to other entities of KGs. This causes such models to miss critical information in tasks where global information is also a significant component for the task, such as in knowledge representation learning. To address the issue, we allow the proper incorporation of global information into the GAT family of models through the use of scaled entity importance, which is calculated by an attention-based global random walk algorithm. In the context of KRL, incorporating global information boosts performance significantly. Experimental results on KG entity prediction against the state-of-thearts sufficiently demonstrate the effectiveness of our proposed model.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. SES: Bridging the Gap Between Explainability and Prediction of Graph Neural NetworksZhenhua Huang, Kunhao Li, Shaojie Wang, Zhaohong Jia et al.ICDE 2024 · 5 citations

Builds on10

  1. Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base EmbeddingsApoorv Saxena, Aditay Tripathi, Partha P. TalukdarACL 2020 · 488 citations
  2. Learning Hierarchy-Aware Knowledge Graph Embeddings for Link PredictionZhanqiu Zhang, Jianyu Cai, Yongdong Zhang, Jie WangAAAI 2020 · 481 citations
  3. GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social MediaYi-Ju Lu, Cheng-Te LiACL 2020 · 387 citations
  4. An Attention-Based Graph Neural Network for Heterogeneous Structural LearningHuiting Hong, Hantao Guo, Yucheng Lin, Xiaoqing Yang et al.AAAI 2020 · 278 citations
  5. Measuring and Improving the Use of Graph Information in Graph Neural NetworksYifan Hou, Jie Zhang, James Cheng, Kaili Ma et al.ICLR 2020 · 148 citations

Related papers

  1. Representation Learning on Knowledge Graphs for Node Importance EstimationHan Huang, Leilei Sun, Bowen Du, Chuanren Liu et al.KDD 2021 · 21 citations
  2. Knowledge-Enhanced Top-K Recommendation in Poincaré BallChen Ma, Liheng Ma, Yingxue Zhang, Haolun Wu et al.AAAI 2021 · 37 citations
  3. Relational Graph Neural Network with Hierarchical Attention for Knowledge Graph CompletionZhao Zhang, Fuzhen Zhuang, Hengshu Zhu, Zhi-Ping Shi et al.AAAI 2020 · 215 citations
  4. Double-Branch Multi-Attention based Graph Neural Network for Knowledge Graph CompletionHongcai Xu, Junpeng Bao, Wenbo LiuACL 2023 · 17 citations
  5. ReInceptionE: Relation-Aware Inception Network with Joint Local-Global Structural Information for Knowledge Graph EmbeddingZhiwen Xie, Guangyou Zhou, Jin Liu, Jimmy Xiangji HuangACL 2020 · 64 citations
Dusk over the sea between two cliffs drawn in fine vertical lines