Relation-Aware Multimodal Analogical Reasoning with Modality Fingerprints and Adaptive Gating
Ruofan Wang, Zijian Huang, Qiqi Wang, Yuchen Su, Robert Amor, Kaiqi Zhao, Meng-Fen Chiang
Abstract
Analogical reasoning over Multimodal Knowledge Graphs (MMKGs) couples abductive relation induction with inductive tail completion. However, existing approaches rely on static fusion mechanisms that overlook the inherent asymmetry of modal relevance: while visual cues elucidate concrete entities, they are often noisy or irrelevant for abstract concepts, where text and graph structure provide decisive signals. Furthermore, prior methods fail to enforce consistency between induced relations and the modality patterns implied by the analogical context. To bridge this gap, we introduce RMAR, a Relation-aware Multimodal Analogical Reasoning framework with two complementary paths. An explicit path estimates modality fingerprints to score compatibility during relation induction and guide fusion during tail completion. An implicit path employs adaptive gating to blend structural, textual, and visual signals conditioned on the specific query context. To address the limitations of current benchmarks, which overrepresent concrete entities, we release MCNetAnalogy, and its companion graph, MCNetKG, a rigorous dataset enriched with abstract concepts and actions. RMAR is backbone-agnostic and works with multimodal knowledge graph embedding (MKGE) and transformer-based (MPT) pipelines. Extensive experiments demonstrate that RMAR delivers consistent gains across both embedding-based and transformer-based backbones, achieving a 29% relative improvement on MCNetAnalogy. Ablation studies confirm that RMAR's relation-aware modulation is particularly effective when modal evidence is weak or ambiguous.
Ask about this paper
Ask your agent about it.
Lune has read the top-tier papers around this one, so every answer names the papers it rests on.
Related papers
- Multimodal Analogical Reasoning over Knowledge GraphsNingyu Zhang, Lei Li, Xiang Chen, Xiaozhuan Liang et al.ICLR 2023 · 10 citations
- Multimodal Reasoning with Multimodal Knowledge GraphJunlin Lee, Yequan Wang, Jing Li, Min ZhangACL 2024 · 29 citations
- Simple Yet Effective: Structure Guided Pre-trained Transformer for Multi-modal Knowledge Graph ReasoningKe Liang, Lingyuan Meng, Yue Liu, Meng Liu et al.ACM MM 2024 · 29 citations
- MMKGR: Multi-hop Multi-modal Knowledge Graph ReasoningShangfei Zheng, Weiqing Wang, Jianfeng Qu, Hongzhi Yin et al.ICDE 2023 · 40 citations
- NativE: Multi-modal Knowledge Graph Completion in the WildYichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu et al.SIGIR 2024 · 39 citations
