Breaking the Noise Barrier: LLM-Guided Semantic Filtering and Enhancement for Multi-Modal Entity Alignment
Chenglong Lu, Chenxiao Li, Jingwei Cheng, Yongquan Ji, Guoqing Chen, Fu Zhang
Abstract
Multi-modal entity alignment (MMEA) aims to identify equivalent entities between two multimodal knowledge graphs (MMKGs). Existing methods have made substantial advancements in enhancing multi-modal fusion. However, the intrinsic noise within modalities, such as the inconsistency in visual modality and redundant attributes, has not been thoroughly investigated. Excessive noise not only weakens semantic representation but also increases the risk of overfitting in attention-based fusion methods. To address this, we propose LGEA (LLM-Guided Entity Alignment), a novel LLM-guided MMEA framework that prioritizes noise reduction before fusion. Specifically, LGEA introduces two key strategies: (1) fine-grained visual filtering to remove irrelevant images at the semantic level, and (2) contextual summarization of attribute information to enhance entity semantics. To our knowledge, we are the first work to apply LLMs for both visual filtering and attribute-level semantic enhancement in MMEA. Experiments on multiple benchmarks, including the noisy FBYG dataset, show that LGEA sets a new state-of-the-art (SOTA) in robust multi-modal alignment, highlighting the potential of noiseaware strategies as a promising direction for future MMEA research 1 .
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Cited by top-tier papers1
Ask how each one uses itBuilds on5
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and GenerationJunnan Li, Dongxu Li, Caiming Xiong, Steven C. H. HoiICML 2022 · 6,549 citations
- Visual Pivoting for (Unsupervised) Entity AlignmentFangyu Liu, Muhao Chen, Dan Roth, Nigel CollierAAAI 2021 · 159 citations
- AFDGCF: Adaptive Feature De-correlation Graph Collaborative Filtering for RecommendationsWei Wu, Chao Wang, Dazhong Shen, Chuan Qin et al.SIGIR 2024 · 33 citations
- Tackling Uncertain Correspondences for Multi-Modal Entity AlignmentLiyi Chen, Ying Sun, Shengzhe Zhang, Yuyang Ye et al.NeurIPS 2024 · 20 citations
- SimDiff: Simple Denoising Probabilistic Latent Diffusion Model for Data Augmentation on Multi-modal Knowledge GraphRan Li, Shimin Di, Lei Chen, Xiaofang ZhouKDD 2024 · 5 citations
Related papers
- Attribute-Consistent Knowledge Graph Representation Learning for Multi-Modal Entity AlignmentQian Li, Shu Guo, Yangyifei Luo, Cheng Ji et al.WWW 2023 · 56 citations
- Learning with Dual-level Noisy Correspondence for Multi-modal Entity AlignmentHaobin Li, Yijie Lin, Peng Hu, Mouxing Yang et al.ICLR 2026 · 2 citations
- Towards Semantic Consistency: Dirichlet Energy Driven Robust Multi-Modal Entity AlignmentYuanyi Wang, Haifeng Sun, Jiabo Wang, Jingyu Wang et al.ICDE 2024 · 13 citations
- IBMEA: Exploring Variational Information Bottleneck for Multi-modal Entity AlignmentTaoyu Su, Jiawei Sheng, Shicheng Wang, Xinghua Zhang et al.ACM MM 2024 · 7 citations
- Pseudo-Label Calibration Semi-supervised Multi-Modal Entity AlignmentLuyao Wang, Pengnian Qi, Xigang Bao, Chunlai Zhou et al.AAAI 2024 · 21 citations
