Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion
Zhiqiang Liu, Yichi Zhang, Mengshu Sun, Lei Liang, Wen Zhang
摘要
Multi-modal knowledge graph completion (MMKGC) aims to discover missing facts in multi-modal knowledge graphs (MMKGs) by leveraging both structural relationships and diverse modality information of entities. Existing MMKGC methods follow two multi-modal paradigms: fusion-based and ensemble-based. Fusion-based methods employ fixed fusion strategies, which inevitably leads to the loss of modality-specific information and a lack of flexibility to adapt to varying modality relevance across contexts. In contrast, ensemble-based methods retain modality independence through dedicated sub-models but struggle to capture the nuanced, context-dependent semantic interplay between modalities. To overcome these dual limitations, we propose a novel MMKGC method M-Hyper, which achieves the coexistence and collaboration of fused and independent modality representations. Our method integrates the strengths of both paradigms, enabling effective cross-modal interactions while maintaining modality-specific information. Inspired by "quaternion" algebra, we utilize its four orthogonal bases to represent multiple independent modalities and employ the Hamilton product to efficiently model pair-wise interactions among them. Specifically, we introduce a Fine-grained Entity Representation Factorization (FERF) module and a Robust Relation-aware Modality Fusion (R2MF) module to obtain robust representations for three independent modalities and one fused modality. The resulting four modality representations are then mapped to the four orthogonal bases of a biquaternion for comprehensive modality interaction. Extensive experiments indicate its state-of-the-art performance with better robustness. Our dataset and code are available at https://github.com/zjukg/M-Hyper .
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper13
- Dual Quaternion Knowledge Graph EmbeddingsZongsheng Cao, Qianqian Xu, Zhiyong Yang, Xiaochun Cao 等AAAI 2021 · 被引用 186 次
- OTKGE: Multi-modal Knowledge Graph Embeddings via Optimal TransportZongsheng Cao, Qianqian Xu, Zhiyong Yang, Yuan He 等NeurIPS 2022 · 被引用 117 次
- IMF: Interactive Multimodal Fusion Model for Link PredictionXinhang Li, Xiangyu Zhao, Jiaxing Xu, Yong Zhang 等WWW 2023 · 被引用 113 次
- Relation-enhanced Negative Sampling for Multimodal Knowledge Graph CompletionDerong Xu, Tong Xu, Shiwei Wu, Jingbo Zhou 等ACM MM 2022 · 被引用 89 次
- TIVA-KG: A Multimodal Knowledge Graph with Text, Image, Video and AudioXin Wang, Benyuan Meng, Hong Chen, Yuan Meng 等ACM MM 2023 · 被引用 71 次
相关 Paper
- NativE: Multi-modal Knowledge Graph Completion in the WildYichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu 等SIGIR 2024 · 被引用 39 次
- Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation LearningYichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu 等ICLR 2025 · 被引用 1 次
- Mixed-Curvature Multi-Modal Knowledge Graph CompletionYuxiao Gao, Fuwei Zhang, Zhao Zhang, Xiaoshuang Min 等AAAI 2025 · 被引用 5 次
- MoSE: Modality Split and Ensemble for Multimodal Knowledge Graph CompletionYu Zhao, Xiangrui Cai, Yike Wu, Haiwei Zhang 等EMNLP 2022 · 被引用 66 次
- HFR-MKGC: Hierarchical Fusion Reasoning with MLLMs for Multi-modal Knowledge Graph CompletionDi Wang, Junping Du, Zhe Xue, Meiyu Liang 等AAAI 2026
