Lune

EMNLP Findings2024

MM-ChatAlign: A Novel Multimodal Reasoning Framework based on Large Language Models for Entity Alignment

Xuhui Jiang, Yinghan Shen, Zhichao Shi, Chengjin Xu, Wei Li, Huang Zihe, Jian Guo, Yuanzhuo Wang

Abstract

Multimodal entity alignment (MMEA) integrates multi-source and cross-modal knowledge graphs, a crucial yet challenging task for data-centric applications. Traditional MMEA methods derive the visual embeddings of entities and combine them with other modal data for alignment by embedding similarity comparison. However, these methods are hampered by the limited comprehension of visual attributes and deficiencies in realizing and bridging the semantics of multimodal data. To address these challenges, we propose MM-ChatAlign, a novel framework that utilizes the visual reasoning abilities of MLLMs for MMEA. The framework features an embedding-based candidate collection module that adapts to various knowledge representation strategies, effectively filtering out irrelevant reasoning candidates. Additionally, a reasoning and rethinking module, powered by MLLMs, enhances alignment by efficiently utilizing multimodal information. Extensive experiments on four MMEA datasets demonstrate MM-ChatAlign's superiority and underscore the significant potential of MLLMs in MMEA tasks. The source code is available at https://github.com/jxh4945777/MMEA/.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Implicit Fine-tuning via Context Engineering: A Curriculum Learning Framework for Multimodal Entity Alignment cites Learning Transferable Visual Models From Natural Language Supervision
  • Implicit Fine-tuning via Context Engineering: A Curriculum Learning Framework for Multimodal Entity Alignment cites Boosting the Speed of Entity Alignment 10 ×: Dual Attention Matching Network with Normalized Hard Sample Mining
  • Implicit Fine-tuning via Context Engineering: A Curriculum Learning Framework for Multimodal Entity Alignment cites Visual Pivoting for (Unsupervised) Entity Alignment
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Taming the Untamed: Graph-Based Knowledge Retrieval and Reasoning for MLLMs to Conquer the UnknownBowen Wang, Zhouqiang Jiang, Yasuaki Susumu, Shotaro Miwa et al.ICCV 2025 · 1 citation
  2. Implicit Fine-tuning via Context Engineering: A Curriculum Learning Framework for Multimodal Entity AlignmentYunpeng Hong, Chenyang Bu, Di Wu, Yi He et al.KDD 2026

Builds on13

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. Diachronic Embedding for Temporal Knowledge Graph CompletionRishab Goel, Seyed Mehran Kazemi, Marcus A. Brubaker, Pascal PoupartAAAI 2020 · 423 citations
  3. A Benchmarking Study of Embedding-based Entity Alignment for Knowledge GraphsZequn Sun, Qingheng Zhang, Wei Hu, Chengming Wang et al.VLDB 2020 · 297 citations
  4. Visual Pivoting for (Unsupervised) Entity AlignmentFangyu Liu, Muhao Chen, Dan Roth, Nigel CollierAAAI 2021 · 159 citations
  5. Boosting the Speed of Entity Alignment 10 ×: Dual Attention Matching Network with Normalized Hard Sample MiningXin Mao, Wenting Wang, Yuanbin Wu, Man LanWWW 2021 · 148 citations

Related papers

  1. Cross-Modal Graph Attention Network for Entity AlignmentBaogui Xu, Chengjin Xu, Bing SuACM MM 2023 · 21 citations
  2. Unlocking the Power of Large Language Models for Entity AlignmentXuhui Jiang, Yinghan Shen, Zhichao Shi, Chengjin Xu et al.ACL 2024 · 17 citations
  3. Multi-modal Siamese Network for Entity AlignmentLiyi Chen, Zhi Li, Tong Xu, Han Wu et al.KDD 2022 · 82 citations
  4. Breaking the Noise Barrier: LLM-Guided Semantic Filtering and Enhancement for Multi-Modal Entity AlignmentChenglong Lu, Chenxiao Li, Jingwei Cheng, Yongquan Ji et al.EMNLP 2025
  5. Attribute-Consistent Knowledge Graph Representation Learning for Multi-Modal Entity AlignmentQian Li, Shu Guo, Yangyifei Luo, Cheng Ji et al.WWW 2023 · 56 citations
Dusk over the sea between two cliffs drawn in fine vertical lines