Multi-Granularity Multi-Modal Knowledge Graph Representation Learning via Subgraph-Aware Adaptive Fusion and Hierarchical Relation Modeling
Peining Li, Meiyu Liang, Wei Huang, Junping Du, Zhe Xue, Guanhua Ye, Wu Liu, Lei Shi
摘要
Multi-modal knowledge graphs (MMKGs) enrich traditional knowledge graphs by incorporating heterogeneous modalities such as textual descriptions and visual content, offering complementary semantic cues for knowledge reasoning. However, existing approaches often overlook the structural dependencies within each modality, apply static or coarse-grained fusion strategies, and insufficiently model relational semantics. We propose a Multi-Granularity Multi-Modal Knowledge Graph Representation Learning Method via Subgraph-aware Adaptive Fusion and Hierarchical Relation Modeling (SAFER ), which implement multi-modal knowledge representation through adaptive fusion of multi-granularity information such as multi-modal semantics, knowledge structures and relations. SAFER explicitly constructs modality-specific subgraphs and employs structure-aware graph attention networks to effectively capture intra-modal structural dependencies. We propose an adaptive multi-modal fusion mechanism, which aggregates modality-specific embeddings at the semantic level by dynamically assigning entity-specific modality weights. We design a two-stage multi-granularity knowledge relation modeling strategy, which utilizes a structure-aware multi-modal adaptive pre-fusion to preserve topological information and a relation-aware graph attention network (RGAT) post-fusion to encode relational semantics. Extensive experiments on several benchmark datasets demonstrate that the proposed SAFER significantly outperforms competitive baselines on link prediction and relation reasoning tasks.
问问这篇 Paper
问问你的智能体。
Lune 读过与它相关的顶会 Paper,每个回答都会注明依据哪几篇。
相关 Paper
- MMKGR: Multi-hop Multi-modal Knowledge Graph ReasoningShangfei Zheng, Weiqing Wang, Jianfeng Qu, Hongzhi Yin 等ICDE 2023 · 被引用 40 次
- Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation LearningYichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu 等ICLR 2025 · 被引用 1 次
- Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph CompletionXiang Chen, Ningyu Zhang, Lei Li, Shumin Deng 等SIGIR 2022 · 被引用 227 次
- Multimodal Reasoning with Multimodal Knowledge GraphJunlin Lee, Yequan Wang, Jing Li, Min ZhangACL 2024 · 被引用 29 次
- DySarl: Dynamic Structure-Aware Representation Learning for Multimodal Knowledge Graph ReasoningKangzheng Liu, Feng Zhao, Yu Yang, Guandong XuACM MM 2024 · 被引用 4 次
