Lune

EMNLP Findings2025

DTDES-KGE: Dual-Teacher Knowledge Distillation with Distinct Embedding Spaces for Knowledge Graph Embeddings

Bofan Wei, Hongyuan Xu, Yuhang Niu, Jiarui Ren, Yanlong Wen, Xiaojie Yuan

Abstract

Knowledge distillation for knowledge graph embedding (KGE) models effectively compresses KGE models by reducing their embedding dimensions. While existing methods distill knowledge from a high-dimensional teacher to a low-dimensional student, they typically rely on a single teacher embedding space, thereby overlooking valuable complementary knowledge from teachers in distinct embedding spaces. This paper introduces DTDES-KGE, a novel knowledge distillation framework that significantly enhances distillation performance by leveraging dual teachers in distinct embedding spaces. To overcome the challenge of spatial heterogeneity when integrating knowledge from dual teachers, we propose a spatial compatibility module for reconciliation. Additionally, we introduce a student-aware knowledge fusion mechanism to fuse the knowledge from dual teachers dynamically. Extensive experiments on two real-world datasets validate the effectiveness of DTDES-KGE.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Ultrahyperbolic Knowledge Graph Embeddings cites Low-Dimensional Hyperbolic Knowledge Graph Embeddings
  • CAKE: A Scalable Commonsense-Aware Framework For Multi-View Knowledge Graph Completion cites Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on16

  1. Learning Hierarchy-Aware Knowledge Graph Embeddings for Link PredictionZhanqiu Zhang, Jianyu Cai, Yongdong Zhang, Jie WangAAAI 2020 · 481 citations
  2. One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge DistillationZhiwei Hao, Jianyuan Guo, Kai Han, Yehui Tang et al.NeurIPS 2023 · 205 citations
  3. CAKE: A Scalable Commonsense-Aware Framework For Multi-View Knowledge Graph CompletionGuanglin Niu, Bo Li, Yongfei Zhang, Shiliang PuACL 2022 · 56 citations
  4. Low-Dimensional Hyperbolic Knowledge Graph EmbeddingsInes Chami, Adva Wolf, Da-Cheng Juan, Frederic Sala et al.ACL 2020 · 48 citations
  5. Ultrahyperbolic Knowledge Graph EmbeddingsBo Xiong, Shichao Zhu, Mojtaba Nayyeri, Chengjin Xu et al.KDD 2022 · 32 citations

Related papers

  1. IterDE: An Iterative Knowledge Distillation Framework for Knowledge Graph EmbeddingsJiajun Liu, Peng Wang, Ziyu Shang, Chenxiao WuAAAI 2023 · 26 citations
  2. Joint Pre-training and Local Re-training: Transferable Representation Learning on Multi-source Knowledge GraphsZequn Sun, Jiacheng Huang, Jinghao Lin, Xiaozhou Xu et al.KDD 2023 · 5 citations
  3. MulDE: Multi-teacher Knowledge Distillation for Low-dimensional Knowledge Graph EmbeddingsKai Wang, Yu Liu, Qian Ma, Quan Z. ShengWWW 2021 · 67 citations
  4. Multi-Scale Distillation from Multiple Graph Neural NetworksChunhai Zhang, Jie Liu, Kai Dang, Wenzheng ZhangAAAI 2022 · 17 citations
  5. Geometry Awakening: Cross-Geometry Learning Exhibits Superiority over Individual StructuresYadong Sun, Xiaofeng Cao, Yu Wang, Wei Ye et al.NeurIPS 2024 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines