ACL Findings2024
AustroTox: A Dataset for Target-Based Austrian German Offensive Language Detection
Pia Pachinger, Janis Goldzycher, Anna Maria Planitzer, Wojciech Kusa, Allan Hanbury, Julia Neidhardt
Abstract
Model interpretability in toxicity detection greatly profits from token-level annotations. However, currently such annotations are only available in English. We introduce a dataset annotated for offensive language detection sourced from a news forum, notable for its incorporation of the Austrian German dialect, comprising 4,562 user comments. In addition to binary offensiveness classification, we identify spans within each comment constituting vulgar language or representing targets of offensive statements. We evaluate fine-tuned language models as well as large language models in a zero-and few-shot fashion. The results indicate that while fine-tuned models excel in detecting linguistic peculiarities such as vulgar dialect, large language models demonstrate superior performance in detecting offensiveness in AustroTox. We publish the data and code 1 .
Content warning: This paper contains examples of offensive language to describe the annotation scheme.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers2
Builds on5
- Toward a Perspectivist Turn in Ground Truthing for Predictive ComputingFederico Cabitza, Andrea Campagner, Valerio BasileAAAI 2023 · 236 citations
- Personalizing Content Moderation on Social Media: User Perspectives on Moderation Choices, Interface Design, and LaborShagun Jhaver, Alice Qian Zhang, Quan Ze Chen, Nikhila Natarajan et al.CSCW 2023 · 87 citations
- Decolonizing Content Moderation: Does Uniform Global Community Standard Resemble Utopian Equality or Western Power Hegemony?Farhana Shahid, Aditya VashisthaCHI 2023 · 63 citations
- Social Bias Frames: Reasoning about Social and Power Implications of LanguageMaarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky et al.ACL 2020 · 16 citations
- The State of Profanity Obfuscation in Natural Language Processing Scientific PublicationsDebora Nozza, Dirk HovyACL Findings 2023 · 4 citations
Related papers
- KOLD: Korean Offensive Language DatasetYounghoon Jeong, Juhyun Oh, Jongwon Lee, Jaimeen Ahn et al.EMNLP 2022 · 41 citations
- HateCOT: An Explanation-Enhanced Dataset for Generalizable Offensive Speech Detection via Large Language ModelsHuy Nghiem, Hal Daumé IIIEMNLP Findings 2024 · 10 citations
- Mapping Toxic Comments Across Demographics: A Dataset from German Public BroadcastingJan Fillies, Michael Peter Hoffmann, Rebecca Reichel, Roman Salzwedel et al.EMNLP 2025
- ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking PerturbationsYunze Xiao, Yujia Hu, Kenny T. W. Choo, Roy Ka-Wei LeeEMNLP 2024 · 5 citations
- µsicalnote Something Just Like TRuST µsicalnote *: Toxicity Recognition of Span and TargetBerk Atil, Namrata Sureddy, Rebecca J. PassonneauACL Findings 2026
