ACL Findings2022
Suum Cuique: Studying Bias in Taboo Detection with a Community Perspective
Osama Khalid, Jonathan Rusert, Padmini Srinivasan
Abstract
Prior research has discussed and illustrated the need to consider linguistic norms at the community level when studying taboo (hateful/offensive/toxic etc.) language. However, a methodology for doing so, that is firmly founded on community language norms is still largely absent. This can lead both to biases in taboo text classification and limitations in our understanding of the causes of bias. We propose a method to study bias in taboo classification and annotation where a community perspective is front and center. This is accomplished by using special classifiers tuned for each community's language. In essence, these classifiers represent community level language norms. We use these to study bias and find, for example, biases are largest against African Americans (7/10 datasets and all 3 classifiers examined). In contrast to previous papers we also study other communities and find, for example, strong biases against South Asians. In a small scale user study we illustrate our key idea which is that common utterances, i.e., those with high alignment scores with a community (community classifier confidence scores) are unlikely to be regarded taboo. Annotators who are community members contradict taboo classification decisions and annotations in a majority of instances. This paper is a significant step toward reducing false positive taboo decisions that over time harm minority communities.
⋆ This paper examines taboo language as a case study. The reader is cautioned that the paper contains strong language.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on2
- SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language IdentificationSara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Marcos Zampieri et al.ACL Findings 2021 · 79 citations
- Language (Technology) is Power: A Critical Survey of "Bias" in NLPSu Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna M. WallachACL 2020 · 68 citations
Related papers
- Analyzing Norm Violations in Live-Stream ChatJihyung Moon, Dong-Ho Lee, Hyundong Cho, Woojeong Jin et al.EMNLP 2023 · 5 citations
- Robustness and Confounders in the Demographic Alignment of LLMs with Human Perceptions of OffensivenessShayan Alipour, Indira Sen, Mattia Samory, Tanushree MitraACL Findings 2025 · 2 citations
- COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive StatementsXuhui Zhou, Hao Zhu, Akhila Yerukola, Thomas Davidson et al.ACL Findings 2023 · 13 citations
- Annotation Sensitivity: Training Data Collection Methods Affect Model PerformanceChristoph Kern, Stephanie Eckman, Jacob Beck, Rob Chew et al.EMNLP Findings 2023 · 5 citations
- Centering the Margins: Outlier-Based Identification of Harmed Populations in Toxicity DetectionVyoma Raman, Eve Fleisig, Dan KleinEMNLP 2023 · 1 citation
