ACL Findings2025
Model-Dependent Moderation: Inconsistencies in Hate Speech Detection Across LLM-based Systems
Abstract
Content moderation systems powered by large language models (LLMs) are increasingly deployed to detect hate speech; however, no systematic comparison exists between different systems. If different systems produce different outcomes for the same content, it undermines consistency and predictability, leading to moderation decisions that appear arbitrary or unfair. Analyzing seven leading models-dedicated Moderation Endpoints (OpenAI, Mistral), frontier LLMs (Claude 3.5 Sonnet, GPT-4o, Mistral Large, DeepSeek V3), and specialized content moderation APIs (Google Perspective API)-we demonstrate that moderation system choice fundamentally determines hate speech classification outcomes. Using a novel synthetic dataset of 1.3+ million sentences from a factorial design, we find identical content receives markedly different classification values across systems, with variations especially pronounced for specific demographic groups. Analysis across 125 distinct groups reveals these divergences reflect systematic differences in how models establish decision boundaries around harmful content, highlighting significant implications for automated content moderation.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers4
- Evaluating Large Language Models for Detecting AntisemitismJay Patel, Hrudayangam Mehta, Jeremy BlackburnEMNLP 2025 · 1 citation
- Tackling Model Bias via Game-theoretic Multi-agent Collaboration Framework for Hateful Meme ClassificationYiwei Wei, Zhengliang Guo, Shaozu Yuan, Chengyin Hu et al.CVPR 2026
- SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMsSihang Zhao, Kangrui Yu, Youliang Yuan, Pinjia He et al.ACL 2026
- When in Doubt, Consult: Expert Debate for Sexism Detection via Confidence-Based RoutingAnwar Alajmi, Gabriele PergolaACL 2026
Builds on3
- Language (Technology) is Power: A Critical Survey of "Bias" in NLPSu Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna M. WallachACL 2020 · 68 citations
- NaijaHate: Evaluating Hate Speech Detection on Nigerian Twitter Using Representative DataManuel Tonneau, Pedro Vitor Quinta de Castro, Karim Lasri, Ibrahim Farouq et al.ACL 2024
- A Survey of Race, Racism, and Anti-Racism in NLPAnjalie Field, Su Lin Blodgett, Zeerak Waseem, Yulia TsvetkovACL 2021
Related papers
- Beyond Hate Speech: NLP's Challenges and Opportunities in Uncovering Dehumanizing LanguageHamidreza Saffari, Mohammadamin Shafiei, Hezhao Zhang, Lasana T. Harris et al.EMNLP 2025
- There is No War in Ba Sing Se: A Global Analysis of Content Moderation in Large Language ModelsFriedemann Lipphardt, Moonis Ali, Martin Banzer, Anja Feldmann et al.NDSS 2026 · 1 citation
- Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic VariationsDavid Hartmann, Amin Oueslati, Dimitri Staufer, Lena Pohlmann et al.CHI 2025 · 37 citations
- Confident, Calibrated, or Complicit: Safety Alignment and Ideological Bias in LLM Hate Speech DetectionSanjeevan Selvaganapathy, Mehwish NasimACL 2026
- Who Speaks Matters: Analysing the Influence of the Speaker's Linguistic Identity on Hate ClassificationAnanya Malik, Kartik Sharma, Shaily Bhatt, Lynnette Hui Xian NgEMNLP Findings 2025
