Lune

EMNLP Findings2024

Large Language Models for Propaganda Span Annotation

Maram Hasanain, Fatema Ahmad, Firoj Alam

Abstract

The use of propagandistic techniques in online content has increased in recent years aiming to manipulate online audiences. Fine-grained propaganda detection and extraction of textual spans where propaganda techniques are used, are essential for more informed content consumption. Automatic systems targeting the task over lower resourced languages are limited, usually obstructed by lack of large scale training datasets. Our study investigates whether Large Language Models (LLMs), such as GPT-4, can effectively extract propagandistic spans. We further study the potential of employing the model to collect more cost-effective annotations. Finally, we examine the effectiveness of labels provided by GPT-4 in training smaller language models for the task. The experiments are performed over a large-scale inhouse manually annotated dataset. The results suggest that providing more annotation context to GPT-4 within prompts improves its performance compared to human annotators. Moreover, when serving as an expert annotator (consolidator), the model provides labels that have higher agreement with expert annotators, and lead to specialized models that achieve stateof-the-art over an unseen Arabic testing set. Finally, our work is the first to show the potential of utilizing LLMs to develop annotated datasets for propagandistic spans detection task prompting it with annotations from human annotators with limited expertise. All scripts and annotations will be shared with the community. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Can Third Parties Read Our Emotions?Jiayi Li, Yingfan Zhou, Pranav Narayanan Venkit, Halima Binte Islam et al.ACL 2025 · 5 citations
  2. KinyaProp: Fine-Grained Propaganda Annotation in KinyarwandaManzi Fabrice Niyigaba, Ivory Yang, Soroush VosoughiACL 2026
  3. Cross-National Information Attacks: A Two-Decade Analysis of Troll Behavior in KoreaJaehong Kim, Hyeonseung Kim, Jiseon Kim, Alice Oh et al.USENIX Security 2026

Builds on4

  1. Large Language Models can Accurately Predict Searcher PreferencesPaul Thomas, Seth Spielman, Nick Craswell, Bhaskar MitraSIGIR 2024 · 153 citations
  2. Is GPT-3 a Good Data Annotator?Bosheng Ding, Chengwei Qin, Linlin Liu, Yew Ken Chia et al.ACL 2023 · 133 citations
  3. MEGA: Multilingual Evaluation of Generative AIKabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng et al.EMNLP 2023 · 91 citations
  4. Holistic Inter-Annotator Agreement and Corpus Coherence Estimation in a Large-scale Multilingual Annotation CampaignNicolas Stefanovitch, Jakub PiskorskiEMNLP 2023 · 1 citation

Related papers

  1. PropXplain: Can LLMs Enable Explainable Propaganda Detection?Maram Hasanain, Md. Arid Hasan, Mohamed Bayan Kmainasi, Elisa Sartori et al.EMNLP Findings 2025
  2. Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation SchemasLukas Stähelin, Veronika Solopova, Max Upravitelev, David Kaplan et al.ACL Findings 2026
  3. HQP: A Human-Annotated Dataset for Detecting Online PropagandaAbdurahman Maarouf, Dominik Bär, Dominique Geissler, Stefan FeuerriegelACL Findings 2024 · 7 citations
  4. Detecting Propaganda Techniques in Code-Switched Social Media TextMuhammad Umar Salman, Asif Hanif, Shady Shehata, Preslav NakovEMNLP 2023 · 5 citations
  5. On the Generalization of Training-based ChatGPT Detection MethodsHan Xu, Jie Ren, Pengfei He, Shenglai Zeng et al.EMNLP Findings 2024 · 9 citations
Dusk over the sea between two cliffs drawn in fine vertical lines