Lune

ACL Findings2023

Playing the Part of the Sharp Bully: Generating Adversarial Examples for Implicit Hate Speech Detection

Nicolás Benjamín Ocampo, Elena Cabrio, Serena Villata

Abstract

Research on abusive content detection on social media has primarily focused on explicit forms of hate speech (HS), that are often identifiable by recognizing hateful words and expressions. Messages containing linguistically subtle and implicit forms of hate speech still constitute an open challenge for automatic hate speech detection. In this paper, we propose a new framework for generating adversarial implicit HS short-text messages using Auto-regressive Language Models. Moreover, we propose a strategy to group the generated implicit messages by their complexity levels (EASY, MEDIUM, and HARD categories) characterizing how challenging these messages are for supervised classifiers. Finally, relying on (Dinan et al., 2019;Vidgen et al., 2021), we propose a "build it, break it, fix it", training scheme using HARD messages showing how iteratively retraining on HARD messages substantially leverages SOTA models' performances on implicit HS benchmarks.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

6 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • ImpScore: A Learnable Metric For Quantifying The Implicitness Level of Sentences cites Latent Hatred: A Benchmark for Understanding Implicit Hate Speech
  • AmpleHate: Amplifying the Attention for Versatile Implicit Hate Detection cites Latent Hatred: A Benchmark for Understanding Implicit Hate Speech
  • ImpliHateVid: A Benchmark Dataset and Two-stage Contrastive Learning Framework for Implicit Hate Speech Detection in Videos cites Latent Hatred: A Benchmark for Understanding Implicit Hate Speech
  • Don't Go To Extremes: Revealing the Excessive Sensitivity and Calibration Limitations of LLMs in Implicit Hate Speech Detection cites Latent Hatred: A Benchmark for Understanding Implicit Hate Speech
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers6

  1. What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot DetectionShangbin Feng, Herun Wan, Ningnan Wang, Zhaoxuan Tan et al.ACL 2024 · 19 citations
  2. ImpliHateVid: A Benchmark Dataset and Two-stage Contrastive Learning Framework for Implicit Hate Speech Detection in VideosMohammad Zia Ur Rehman, Anukriti Bhatnagar, Omkar Kabde, Shubhi Bansal et al.ACL 2025 · 11 citations
  3. AmpleHate: Amplifying the Attention for Versatile Implicit Hate DetectionYejin Lee, Joonghyuk Hahn, Hyeseon Ahn, Yo-Sub HanEMNLP 2025 · 2 citations
  4. ImpScore: A Learnable Metric For Quantifying The Implicitness Level of SentencesYuxin Wang, Xiaomeng Zhu, Weimin Lyu, Saeed Hassanpour et al.ICLR 2025
  5. Don't Go To Extremes: Revealing the Excessive Sensitivity and Calibration Limitations of LLMs in Implicit Hate Speech DetectionMin Zhang, Jianfeng He, Taoran Ji, Chang-Tien LuACL 2024

Builds on12

  1. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language ModelsSamuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi et al.EMNLP Findings 2020 · 692 citations
  2. Adversarial NLI: A New Benchmark for Natural Language UnderstandingYixin Nie, Adina Williams, Emily Dinan, Mohit Bansal et al.ACL 2020 · 602 citations
  3. BERT-ATTACK: Adversarial Attack Against BERT Using BERTLinyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue et al.EMNLP 2020 · 529 citations
  4. Latent Hatred: A Benchmark for Understanding Implicit Hate SpeechMai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi et al.EMNLP 2021 · 159 citations
  5. Improving Generalizability in Implicitly Abusive Language Detection with Concept Activation VectorsIsar Nejadgholi, Kathleen C. Fraser, Svetlana KiritchenkoACL 2022 · 27 citations

Related papers

  1. Unmasking the Hidden Meaning: Bridging Implicit and Explicit Hate Speech Embedding RepresentationsNicolás Benjamín Ocampo, Elena Cabrio, Serena VillataEMNLP Findings 2023 · 6 citations
  2. HARE: Explainable Hate Speech Detection with Step-by-Step ReasoningYongjin Yang, Joonkee Kim, Yujin Kim, Namgyu Ho et al.EMNLP Findings 2023 · 24 citations
  3. Fight Fire with Fire: Fine-tuning Hate Detectors using Large Samples of Generated Hate SpeechTomer Wullach, Amir Adler, Einat MinkovEMNLP Findings 2021 · 33 citations
  4. HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate CampaignsXinyue Shen, Yixin Wu, Yiting Qu, Michael Backes et al.USENIX Security 2025
  5. ConPrompt: Pre-training a Language Model with Machine-Generated Data for Implicit Hate Speech DetectionYoungwook Kim, Shinwoo Park, Youngsoo Namgoong, Yo-Sub HanEMNLP Findings 2023 · 11 citations
Dusk over the sea between two cliffs drawn in fine vertical lines