ACL Findings2022
Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Study
Serra Sinem Tekiroglu, Helena Bonaldi, Margherita Fanton, Marco Guerini
Abstract
In this work, we present an extensive study on the use of pre-trained language models for the task of automatic Counter Narrative (CN) generation to fight online hate speech in English. We first present a comparative study to determine whether there is a particular Language Model (or class of LMs) and a particular decoding mechanism that are the most appropriate to generate CNs. Findings show that autoregressive models combined with stochastic decodings are the most promising. We then investigate how an LM performs in generating a CN with regard to an unseen target of hate. We find out that a key element for successful 'out of target' experiments is not an overall similarity with the training data but the presence of a specific subset of training data, i. e. a target that shares some commonalities with the test target that can be defined a-priori. We finally introduce the idea of a pipeline based on the addition of an automatic post-editing step to refine generated CNs.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
5 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation cites RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
- Outcome-Constrained Large Language Models for Countering Hate Speech cites Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech
- Fact-based Counter Narrative Generation to Combat Hate Speech cites Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech
- Fact-based Counter Narrative Generation to Combat Hate Speech cites Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering
- Fact-based Counter Narrative Generation to Combat Hate Speech cites Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech
- Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering cites Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech
- Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering cites Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech
- Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering cites The Curious Case of Neural Text Degeneration
- Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech cites RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
- Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech cites The Curious Case of Neural Text Degeneration
- RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models cites The Curious Case of Neural Text Degeneration
- Is Safer Better? The Impact of Guardrails on the Argumentative Strength of LLMs in Hate Speech Countering cites Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering
- Is Safer Better? The Impact of Guardrails on the Argumentative Strength of LLMs in Hate Speech Countering cites Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech
Cited by top-tier papers5
- Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech CounteringHelena Bonaldi, Sara Dellantonio, Serra Sinem Tekiroglu, Marco GueriniEMNLP 2022 · 18 citations
- Fact-based Counter Narrative Generation to Combat Hate SpeechBrian Wilk, Homaira Huda Shomee, Suman Kalyan Maity, Sourav MedyaWWW 2025 · 6 citations
- Outcome-Constrained Large Language Models for Countering Hate SpeechLingzi Hong, Pengcheng Luo, Eduardo Blanco, Xiaoying SongEMNLP 2024 · 5 citations
- Is Safer Better? The Impact of Guardrails on the Argumentative Strength of LLMs in Hate Speech CounteringHelena Bonaldi, Greta Damo, Nicolás Benjamín Ocampo, Elena Cabrio et al.EMNLP 2024 · 2 citations
- Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust EvaluationHuije Lee, Jisu Shin, Hoyun Song, Changgeon Ko et al.ACL 2026
Builds on8
- The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language ModelsSamuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi et al.EMNLP Findings 2020 · 692 citations
- Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate SpeechWanzheng Zhu, Suma BhatACL Findings 2021 · 31 citations
- Towards Knowledge-Grounded Counter Narrative Generation for Hate SpeechYi-Ling Chung, Serra Sinem Tekiroglu, Marco GueriniACL Findings 2021 · 29 citations
Related papers
- High-quality argumentative information in low resources approaches improve counter-narrative generationDamián Ariel Furman, Pablo Torres, José A. Rodríguez, Diego Letzen et al.EMNLP Findings 2023 · 3 citations
- Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate SpeechMargherita Fanton, Helena Bonaldi, Serra Sinem Tekiroglu, Marco GueriniACL 2021
- Fight Fire with Fire: Fine-tuning Hate Detectors using Large Samples of Generated Hate SpeechTomer Wullach, Amir Adler, Einat MinkovEMNLP Findings 2021 · 33 citations
- A LLM-based Ranking Method for the Evaluation of Automatic Counter-Narrative GenerationIrune Zubiaga, Aitor Soroa, Rodrigo AgerriEMNLP Findings 2024 · 6 citations
- Generating Counter Narratives against Online Hate Speech: Data and StrategiesSerra Sinem Tekiroglu, Yi-Ling Chung, Marco GueriniACL 2020 · 13 citations
