Relation-Aware Multimodal Analogical Reasoning with Modality Fingerprints and Adaptive Gating
Ruofan Wang, Zijian Huang, Qiqi Wang, Yuchen Su, Robert Amor, Kaiqi Zhao, Meng-Fen Chiang
摘要
Analogical reasoning over Multimodal Knowledge Graphs (MMKGs) couples abductive relation induction with inductive tail completion. However, existing approaches rely on static fusion mechanisms that overlook the inherent asymmetry of modal relevance: while visual cues elucidate concrete entities, they are often noisy or irrelevant for abstract concepts, where text and graph structure provide decisive signals. Furthermore, prior methods fail to enforce consistency between induced relations and the modality patterns implied by the analogical context. To bridge this gap, we introduce RMAR, a Relation-aware Multimodal Analogical Reasoning framework with two complementary paths. An explicit path estimates modality fingerprints to score compatibility during relation induction and guide fusion during tail completion. An implicit path employs adaptive gating to blend structural, textual, and visual signals conditioned on the specific query context. To address the limitations of current benchmarks, which overrepresent concrete entities, we release MCNetAnalogy, and its companion graph, MCNetKG, a rigorous dataset enriched with abstract concepts and actions. RMAR is backbone-agnostic and works with multimodal knowledge graph embedding (MKGE) and transformer-based (MPT) pipelines. Extensive experiments demonstrate that RMAR delivers consistent gains across both embedding-based and transformer-based backbones, achieving a 29% relative improvement on MCNetAnalogy. Ablation studies confirm that RMAR's relation-aware modulation is particularly effective when modal evidence is weak or ambiguous.
问问这篇 Paper
问问你的智能体。
Lune 读过与它相关的顶会 Paper,每个回答都会注明依据哪几篇。
相关 Paper
- Multimodal Analogical Reasoning over Knowledge GraphsNingyu Zhang, Lei Li, Xiang Chen, Xiaozhuan Liang 等ICLR 2023 · 被引用 10 次
- Multimodal Reasoning with Multimodal Knowledge GraphJunlin Lee, Yequan Wang, Jing Li, Min ZhangACL 2024 · 被引用 29 次
- Simple Yet Effective: Structure Guided Pre-trained Transformer for Multi-modal Knowledge Graph ReasoningKe Liang, Lingyuan Meng, Yue Liu, Meng Liu 等ACM MM 2024 · 被引用 29 次
- MMKGR: Multi-hop Multi-modal Knowledge Graph ReasoningShangfei Zheng, Weiqing Wang, Jianfeng Qu, Hongzhi Yin 等ICDE 2023 · 被引用 40 次
- NativE: Multi-modal Knowledge Graph Completion in the WildYichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu 等SIGIR 2024 · 被引用 39 次
