Lune

ACL Findings2026

Prompting Across Time: Evaluating LLMs on Historical and Contemporary Offensive Language

Sanne Hoeken, Sophie Jasmin Spliethoff, Silke Schwandt, Özge Alaçam, Sina Zarrieß

Abstract

Research on hate speech detection (HSD) has centered on modern data, even though offensive language has a much longer history. This paper presents the first systematic evaluation of instruction-tuned LLMs on Early Modern English invectives, compared with a modern hate-speech benchmark. Our work applies a modular prompt design to measure the contribution of definitional richness, contextual grounding, decision rules and few-shot examples. The results indicate that clearer annotation boundaries in the curated historical corpus lead to higher classification performance compared to the modern benchmark, despite the disadvantage of linguistic unfamiliarity. Prompt brittleness, however, persists across both domains. Classification-oriented components (rules, examples) drive the strongest effects, while definitional or contextual additions matter less. Finetuned encoder models still outperform LLMs, but some prompt configurations can narrow the gap. Overall, our study provides practical guidance for prompt design in both digital humanities and HSD and new opportunities for tracing the historical development of hate speech. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • fBERT: A Neural Transformer for Identifying Offensive Content cites Unsupervised Cross-lingual Representation Learning at Scale
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on4

  1. Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
  2. fBERT: A Neural Transformer for Identifying Offensive ContentDiptanu Sarkar, Marcos Zampieri, Tharindu Ranasinghe, Alexander G. Ororbia IIEMNLP Findings 2021 · 43 citations
  3. Probing LLMs for hate speech detection: strengths and vulnerabilitiesSarthak Roy, Ashish Harshavardhan, Animesh Mukherjee, Punyajoy SahaEMNLP Findings 2023 · 23 citations
  4. A Survey of Uncertainty Estimation Methods on Large Language ModelsZhiqiu Xia, Jinxuan Xu, Yuqian Zhang, Hang LiuACL Findings 2025 · 16 citations

Related papers

  1. SAHSD: Enhancing Hate Speech Detection in LLM-Powered Web Applications via Sentiment Analysis and Few-Shot LearningYulong Wang, Hong Li, Ni WeiWWW 2025 · 2 citations
  2. Who Speaks Matters: Analysing the Influence of the Speaker's Linguistic Identity on Hate ClassificationAnanya Malik, Kartik Sharma, Shaily Bhatt, Lynnette Hui Xian NgEMNLP Findings 2025
  3. HateCOT: An Explanation-Enhanced Dataset for Generalizable Offensive Speech Detection via Large Language ModelsHuy Nghiem, Hal Daumé IIIEMNLP Findings 2024 · 10 citations
  4. Legally Enforceable Hate Speech Detection for Public ForumsChu Fei Luo, Rohan Bhambhoria, Samuel Dahan, Xiaodan ZhuEMNLP Findings 2023 · 9 citations
  5. Evaluating Large Language Models for Detecting AntisemitismJay Patel, Hrudayangam Mehta, Jeremy BlackburnEMNLP 2025 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines