Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion
Zhiqiang Liu, Yichi Zhang, Mengshu Sun, Lei Liang, Wen Zhang
Abstract
Multi-modal knowledge graph completion (MMKGC) aims to discover missing facts in multi-modal knowledge graphs (MMKGs) by leveraging both structural relationships and diverse modality information of entities. Existing MMKGC methods follow two multi-modal paradigms: fusion-based and ensemble-based. Fusion-based methods employ fixed fusion strategies, which inevitably leads to the loss of modality-specific information and a lack of flexibility to adapt to varying modality relevance across contexts. In contrast, ensemble-based methods retain modality independence through dedicated sub-models but struggle to capture the nuanced, context-dependent semantic interplay between modalities. To overcome these dual limitations, we propose a novel MMKGC method M-Hyper, which achieves the coexistence and collaboration of fused and independent modality representations. Our method integrates the strengths of both paradigms, enabling effective cross-modal interactions while maintaining modality-specific information. Inspired by "quaternion" algebra, we utilize its four orthogonal bases to represent multiple independent modalities and employ the Hamilton product to efficiently model pair-wise interactions among them. Specifically, we introduce a Fine-grained Entity Representation Factorization (FERF) module and a Robust Relation-aware Modality Fusion (R2MF) module to obtain robust representations for three independent modalities and one fused modality. The resulting four modality representations are then mapped to the four orthogonal bases of a biquaternion for comprehensive modality interaction. Extensive experiments indicate its state-of-the-art performance with better robustness. Our dataset and code are available at https://github.com/zjukg/M-Hyper .
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext e0173309-025f-4e8c-85bf-933223ca6ab2Builds on13
- Dual Quaternion Knowledge Graph EmbeddingsZongsheng Cao, Qianqian Xu, Zhiyong Yang, Xiaochun Cao et al.AAAI 2021 · 186 citations
- OTKGE: Multi-modal Knowledge Graph Embeddings via Optimal TransportZongsheng Cao, Qianqian Xu, Zhiyong Yang, Yuan He et al.NeurIPS 2022 · 117 citations
- IMF: Interactive Multimodal Fusion Model for Link PredictionXinhang Li, Xiangyu Zhao, Jiaxing Xu, Yong Zhang et al.WWW 2023 · 113 citations
- Relation-enhanced Negative Sampling for Multimodal Knowledge Graph CompletionDerong Xu, Tong Xu, Shiwei Wu, Jingbo Zhou et al.ACM MM 2022 · 89 citations
- TIVA-KG: A Multimodal Knowledge Graph with Text, Image, Video and AudioXin Wang, Benyuan Meng, Hong Chen, Yuan Meng et al.ACM MM 2023 · 71 citations
Related papers
- NativE: Multi-modal Knowledge Graph Completion in the WildYichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu et al.SIGIR 2024 · 39 citations
- Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation LearningYichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu et al.ICLR 2025 · 1 citation
- Mixed-Curvature Multi-Modal Knowledge Graph CompletionYuxiao Gao, Fuwei Zhang, Zhao Zhang, Xiaoshuang Min et al.AAAI 2025 · 5 citations
- MoSE: Modality Split and Ensemble for Multimodal Knowledge Graph CompletionYu Zhao, Xiangrui Cai, Yike Wu, Haiwei Zhang et al.EMNLP 2022 · 66 citations
- HFR-MKGC: Hierarchical Fusion Reasoning with MLLMs for Multi-modal Knowledge Graph CompletionDi Wang, Junping Du, Zhe Xue, Meiyu Liang et al.AAAI 2026
