Towards Foundation Models for MMKG: Multi-Task Inductive Generalization via Task-Aware Routing
Shundong Yang, Jing Yang, Xiaowen Jiang, Xiaofen Wang, Laurence T. Yang, Yuan Gao, Xinfa Jiang, Jie Chen, Chaojun Zhang
摘要
Multimodal knowledge graphs (MMKG) extend the representational capacity of conventional knowledge graphs by integrating multimodal contexts. However, despite increasing attention on core tasks—multimodal link prediction (MMLP), multimodal entity alignment (MMEA), and multimodal entity linking (MMEL)—most existing approaches remain confined to meticulously tailored, task-specific and graph-specific paradigms, limiting their practicality in dynamic environments where tasks co-exist and new knowledge graphs continually emerge. In this work, we propose the first unified multi-task inductive inference framework for (MtaIMKG) that can handle multiple core tasks, with universality and transferability to any unseen MMKG, by tackling inherent heterogeneities: i) Feature Heterogeneity: Bridging the semantic gaps in feature representations across domains. ii) Structural Heterogeneity: Overcoming structural divergence in topology across graphs. iii) Task Heterogeneity: Reconciling conflicting objectives and inductive biases across tasks. To address the above challenges, we center on reconciling divergent representation preferences across tasks, delivering only task-pertinent information instead of engaging in indiscriminate aggregation. Specifically, we introduce a task-aware routing network that dynamically integrates and routes multimodal representations, assesses their contributions under task semantics, and produces adaptive, task-conditioned representations for any input graph. In addition, MtaIMKG employs a gated mixture-of-experts to suppress cross-modal noise and extract complementary information across tasks. These strategies tackle the notable heterogeneity challenges in multi-task inductive inference on MMKG, enabling MtaIMKG to effectively zero-shot generalize to any unseen graph. Extensive experiments demonstrate that MtaIMKG achieves SOTA performance in both multi-task inductive and transductive inference with competitive efficiency, confirming its value as a scalable and generalizable solution for MMKG reasoning. Our code is released at https://github.com/MMKGer/MtaIMKG/.
问问这篇 Paper
问问你的智能体。
Lune 读过与它相关的顶会 Paper,每个回答都会注明依据哪几篇。
相关 Paper
- Towards Multimodal Inductive Learning: Adaptively Embedding MMKG via PrototypesShundong Yang, Jing Yang, Xiaowen Jiang, Yuan Gao 等WWW 2025 · 被引用 3 次
- Attribute-Consistent Knowledge Graph Representation Learning for Multi-Modal Entity AlignmentQian Li, Shu Guo, Yangyifei Luo, Cheng Ji 等WWW 2023 · 被引用 56 次
- Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation LearningYichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu 等ICLR 2025 · 被引用 1 次
- MMKGR: Multi-hop Multi-modal Knowledge Graph ReasoningShangfei Zheng, Weiqing Wang, Jianfeng Qu, Hongzhi Yin 等ICDE 2023 · 被引用 40 次
- Towards Foundation Models for Knowledge Graph ReasoningMikhail Galkin, Xinyu Yuan, Hesham Mostafa, Jian Tang 等ICLR 2024 · 被引用 95 次
