Lune

EMNLP Findings2023

Multimodal Automated Fact-Checking: A Survey

Mubashara Akhtar, Michael Sejr Schlichtkrull, Zhijiang Guo, Oana Cocarascu, Elena Simperl, Andreas Vlachos

Abstract

Misinformation is often conveyed in multiple modalities, e.g. a miscaptioned image. Multimodal misinformation is perceived as more credible by humans, and spreads faster than its text-only counterparts. While an increasing body of research investigates automated fact-checking (AFC), previous surveys mostly focus on text. In this survey, we conceptualise a framework for AFC including subtasks unique to multimodal misinformation. Furthermore, we discuss related terms used in different communities and map them to our framework. We focus on four modalities prevalent in real-world fact-checking: text, image, audio, and video. We survey benchmarks and models, and discuss limitations and promising directions for future research.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

14 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking cites HintsOfTruth: A Multimodal Checkworthiness Detection Dataset with Real and Synthetic Claims
  • VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking cites "Image, Tell me your story!" Predicting the original meta-context of visual misinformation
  • HintsOfTruth: A Multimodal Checkworthiness Detection Dataset with Real and Synthetic Claims cites "Image, Tell me your story!" Predicting the original meta-context of visual misinformation
  • "Image, Tell me your story!" Predicting the original meta-context of visual misinformation cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • "Image, Tell me your story!" Predicting the original meta-context of visual misinformation cites Learning Transferable Visual Models From Natural Language Supervision
  • Learning Transferable Visual Models From Natural Language Supervision cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers14

  1. Knowledge Conflicts for LLMs: A SurveyRongwu Xu, Zehan Qi, Zhijiang Guo, Cunxiang Wang et al.EMNLP 2024 · 38 citations
  2. Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-CheckingGreta Warren, Irina Shklovski, Isabelle AugensteinCHI 2025 · 15 citations
  3. "Image, Tell me your story!" Predicting the original meta-context of visual misinformationJonathan Tonglet, Marie-Francine Moens, Iryna GurevychEMNLP 2024 · 6 citations
  4. HintsOfTruth: A Multimodal Checkworthiness Detection Dataset with Real and Synthetic ClaimsMichiel van der Meer, Pavel Korshunov, Sébastien Marcel, Lonneke van der PlasACL 2025 · 5 citations
  5. VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-CheckingMark Rothermel, Marcus Kornmann, Marcus Rohrbach, Anna RohrbachACL 2026 · 4 citations

Builds on30

  1. Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
  2. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  3. High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
  4. FSGAN: Subject Agnostic Face Swapping and ReenactmentYuval Nirkin, Yosi Keller, Tal HassnerICCV 2019 · 710 citations
  5. Few-Shot Adversarial Learning of Realistic Neural Talking Head ModelsEgor Zakharov, Aliaksandra Shysheya, Egor Burkov, Victor S. LempitskyICCV 2019 · 687 citations

Related papers

  1. The Influence of Content Modality on Perceptions of Online MisinformationSuwani Gunasekara, Saumya Pareek, Ryan M. Kelly, Jorge GonçalvesCHI 2025 · 3 citations
  2. Combating Online Misinformation Videos: Characterization, Detection, and Future DirectionsYuyan Bu, Qiang Sheng, Juan Cao, Peng Qi et al.ACM MM 2023 · 38 citations
  3. FakeWorld 1.0: An Omni-modal Benchmark for Fake Media and ContentYifeng Gao, Yifan Ding, Li Wang, Feida Huang et al.ICML 2026
  4. M3D: MultiModal MultiDocument Fine-Grained Inconsistency DetectionChia-Wei Tang, Ting-Chih Chen, Kiet Nguyen, Kazi Sajeed Mehrab et al.EMNLP 2024
  5. Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online ResourcesSahar Abdelnabi, Rakibul Hasan, Mario FritzCVPR 2022 · 79 citations
Dusk over the sea between two cliffs drawn in fine vertical lines