EMNLP Findings2022
Multimodal Knowledge Learning for Named Entity Disambiguation
Dongjie Zhang, Longtao Huang
摘要
With the popularity of online social media, massive-scale multimodal information has brought new challenges to traditional Named Entity Disambiguation (NED) tasks. Recently, Multimodal Named Entity Disambiguation (MNED) has been proposed to link ambiguous mentions with the textual and visual contexts to a predefined knowledge graph. Existing attempts usually perform MNED by annotating multimodal mentions and adding multimodal features to traditional NED models. However, these studies may suffer from 1) failing to model multimodal information at the knowledge level, and 2) lacking multimodal annotation data against the large-scale unlabeled corpus. In this paper, we explore a pioneer study on leveraging multimodal knowledge learning to address the MNED task. Specifically, we first harvest multimodal knowledge in the Meta-Learning way, which is much easier than collecting ambiguous mention corpus. Then we design a knowledge-guided transfer learning strategy to extract unified representation from different modalities. Finally, we propose an Interactive Multimodal Learning Network (IMN) to fully utilize the multimodal information on both the mention and knowledge sides. Extensive experiments conducted on two public MNED datasets demonstrate that the proposed method achieves improvements over the stateof-the-art multimodal methods.
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
已有 3 篇顶会 Paper 引用了它。别等审稿人问起,先弄清它们发现了什么。
脉络
引用它的顶会 Paper3
- DRIN: Dynamic Relation Interactive Network for Multimodal Entity LinkingShangyu Xing, Fei Zhao, Zhen Wu, Chunhui Li 等ACM MM 2023 · 21 次引用
- I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity LinkingZiyan Liu, Junwen Li, Kaiwen Li, Tong Ruan 等ACM MM 2025 · 2 次引用
- Multi-level Mixture of Experts for Multimodal Entity LinkingZhiwei Hu, Víctor Gutiérrez-Basulto, Zhiliang Xiang, Ru Li 等KDD 2025 · 1 次引用
它引用的 Paper5
- VideoBERT: A Joint Model for Video and Language Representation LearningChen Sun, Austin Myers, Carl Vondrick, Kevin Murphy 等ICCV 2019 · 1,396 次引用
- ERNIE-ViL: Knowledge Enhanced Vision-Language Representations through Scene GraphsFei Yu, Jiji Tang, Weichong Yin, Yu Sun 等AAAI 2021 · 414 次引用
- Progressive Semantic Matching for Video-Text RetrievalHongying Liu, Ruyi Luo, Fanhua Shang, Mantang Niu 等ACM MM 2021 · 20 次引用
- Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLPAnthony Chen, Pallavi Gudipati, Shayne Longpre, Xiao Ling 等ACL 2021
- 12-in-1: Multi-Task Vision and Language Representation LearningJiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh 等CVPR 2020
相关 Paper
- Multi-Grained Multimodal Interaction Network for Entity LinkingPengfei Luo, Tong Xu, Shiwei Wu, Chen Zhu 等KDD 2023 · 21 次引用
- Multi-level Matching Network for Multimodal Entity LinkingZhiwei Hu, Víctor Gutiérrez-Basulto, Ru Li, Jeff Z. PanKDD 2025 · 1 次引用
- Grounded Multimodal Named Entity Recognition on Social MediaJianfei Yu, Ziyan Li, Jieming Wang, Rui XiaACL 2023 · 32 次引用
- Multimodal Entity Linking: A New Dataset and A BaselineJingru Gan, Jinchang Luo, Haiwei Wang, Shuhui Wang 等ACM MM 2021 · 41 次引用
- MELOV: Multimodal Entity Linking with Optimized Visual Features in Latent SpaceXuhui Sui, Ying Zhang, Yu Zhao, Kehui Song 等ACL Findings 2024 · 6 次引用
