ACL Findings2021
Incorporating Global Information in Local Attention for Knowledge Representation Learning
Yu Zhao, Han Zhou, Ruobing Xie, Fuzhen Zhuang, Qing Li, Ji Liu
Abstract
Graph Attention Networks (GATs) have proven a promising model that takes advantage of localized attention mechanism to perform knowledge representation learning (KRL) on graph-structure data, e.g., Knowledge Graphs (KGs). While such approaches model entities' local pairwise importance, they lack the capability to model global importance relative to other entities of KGs. This causes such models to miss critical information in tasks where global information is also a significant component for the task, such as in knowledge representation learning. To address the issue, we allow the proper incorporation of global information into the GAT family of models through the use of scaled entity importance, which is calculated by an attention-based global random walk algorithm. In the context of KRL, incorporating global information boosts performance significantly. Experimental results on KG entity prediction against the state-of-thearts sufficiently demonstrate the effectiveness of our proposed model.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers1
Builds on10
- Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base EmbeddingsApoorv Saxena, Aditay Tripathi, Partha P. TalukdarACL 2020 · 488 citations
- Learning Hierarchy-Aware Knowledge Graph Embeddings for Link PredictionZhanqiu Zhang, Jianyu Cai, Yongdong Zhang, Jie WangAAAI 2020 · 481 citations
- GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social MediaYi-Ju Lu, Cheng-Te LiACL 2020 · 387 citations
- An Attention-Based Graph Neural Network for Heterogeneous Structural LearningHuiting Hong, Hantao Guo, Yucheng Lin, Xiaoqing Yang et al.AAAI 2020 · 278 citations
- Measuring and Improving the Use of Graph Information in Graph Neural NetworksYifan Hou, Jie Zhang, James Cheng, Kaili Ma et al.ICLR 2020 · 148 citations
Related papers
- Representation Learning on Knowledge Graphs for Node Importance EstimationHan Huang, Leilei Sun, Bowen Du, Chuanren Liu et al.KDD 2021 · 21 citations
- Knowledge-Enhanced Top-K Recommendation in Poincaré BallChen Ma, Liheng Ma, Yingxue Zhang, Haolun Wu et al.AAAI 2021 · 37 citations
- Relational Graph Neural Network with Hierarchical Attention for Knowledge Graph CompletionZhao Zhang, Fuzhen Zhuang, Hengshu Zhu, Zhi-Ping Shi et al.AAAI 2020 · 215 citations
- Double-Branch Multi-Attention based Graph Neural Network for Knowledge Graph CompletionHongcai Xu, Junpeng Bao, Wenbo LiuACL 2023 · 17 citations
- ReInceptionE: Relation-Aware Inception Network with Joint Local-Global Structural Information for Knowledge Graph EmbeddingZhiwen Xie, Guangyou Zhou, Jin Liu, Jimmy Xiangji HuangACL 2020 · 64 citations
