Lune

ACL Findings2021

Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate

Austin Botelho, Scott Hale, Bertie Vidgen

Abstract

Accurate detection and classification of online hate is a difficult task. Implicit hate is particularly challenging as such content tends to have unusual syntax, polysemic words, and fewer markers of prejudice (e.g., slurs). This problem is heightened with multimodal content, such as memes (combinations of text and images), as they are often harder to decipher than unimodal content (e.g., text alone). This paper evaluates the role of semantic and multimodal context for detecting implicit and explicit hate. We show that both text-and visual-enrichment improves model performance, with the multimodal model (0.771) outperforming other models' F1 scores (0.544, 0.737, and 0.754). While the unimodal-text context-aware (transformer) model was the most accurate on the subtask of implicit hate detection, the multimodal model outperformed it overall because of a lower propensity towards false positives. We find that all models perform better on content with full annotator agreement and that multimodal models are best at classifying the content where annotators disagree. To conduct these investigations, we undertook highquality annotation of a sample of 5,000 multimodal entries. Tweets were annotated for primary category, modality, and strategy. We make this corpus, along with the codebook, code, and final model, freely available.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language ModelsJulia Mendelsohn, Ronan Le Bras, Yejin Choi, Maarten SapACL 2023 · 14 citations
  2. HateDay: Insights from a Global Hate Speech Dataset Representative of a Day on TwitterManuel Tonneau, Diyi Liu, Niyati Malhotra, Scott A. Hale et al.ACL 2025 · 12 citations
  3. Borrowing Human Senses: Comment-Aware Self-Training for Social Media Multimodal ClassificationChunpu Xu, Jing LiEMNLP 2022 · 3 citations

Builds on3

  1. ALBERT: A Lite BERT for Self-supervised Learning of Language RepresentationsZhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel et al.ICLR 2020 · 7,418 citations
  2. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than GeneratorsKevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. ManningICLR 2020 · 541 citations
  3. 12-in-1: Multi-Task Vision and Language Representation LearningJiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh et al.CVPR 2020

Related papers

  1. Disentangling Hate in Online MemesRoy Ka-Wei Lee, Rui Cao, Ziqing Fan, Jing Jiang et al.ACM MM 2021 · 85 citations
  2. The Hateful Memes Challenge: Detecting Hate Speech in Multimodal MemesDouwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami et al.NeurIPS 2020 · 1,022 citations
  3. TOT:Topology-Aware Optimal Transport for Multimodal Hate DetectionLinhao Zhang, Li Jin, Xian Sun, Guangluan Xu et al.AAAI 2023 · 9 citations
  4. Improving Hateful Meme Detection through Retrieval-Guided Contrastive LearningJingbiao Mei, Jinghong Chen, Weizhe Lin, Bill Byrne et al.ACL 2024 · 13 citations
  5. Uncertainty-Guided Modal Rebalance for Hateful Memes DetectionChuanpeng Yang, Yaxin Liu, Fuqing Zhu, Jizhong Han et al.ACL 2024
Dusk over the sea between two cliffs drawn in fine vertical lines