Lune

ACL Findings2026

Propaganda Signals in LLMs: Perspectival Divergence and Narrative Framing in the Russia-Ukraine War

Ofir Shabat, Ido Guy, Kira Radinsky

摘要

Large Language Models (LLMs) are increasingly used to explain, summarize, and translate real-world events, including ongoing geopolitical conflicts. Yet it remains unclear whether they reproduce conflict-specific propaganda and, if so, how this appears in their outputs. We study this question for the Russia-Ukraine war through perspectival divergence, the extent to which model outputs align with competing narratives from different information ecosystems. We construct a conflict-aware evaluation set of neutral English event statements paired with Russian (RU)-and Ukrainian (UA)-oriented reference texts drawn from news outlets and Telegram channels. We then evaluate multiple LLMs under several prompting contexts using a reference-based semantic distance metric that measures directional proximity to RUand UA-oriented references. To explain not only which side a model is closer to but also how that alignment is expressed, we further analyze outputs using five propaganda-relevant categories: Framing & Narrative, Emotional Manipulation, Source & Credibility, Social Pressure & Identity, and Toponymy & Naming. Across models, we find stable, modelspecific leanings and technique profiles that persist across prompts and are not captured by standard factuality-oriented metrics. Our findings show that models that appear neutral under conventional evaluations can still encode systematic, conflict-specific propaganda patterns, underscoring the need for conflict-aware evaluation frameworks.

问 Lune

尽管问,回答会引用原文。

Lune 会读完全文和相关的 Paper,每个回答都附上所依据的原文段落。

可以从这些问题问起

Lune 的依据

全文 · 相关的 Paper

获取答案

免费开始,无需绑卡

它引用的 Paper7

  1. G-Eval: NLG Evaluation using Gpt-4 with Better Human AlignmentYang Liu, Dan Iter, Yichong Xu, Shuohang Wang 等EMNLP 2023 · 549 次引用
  2. Can Large Language Models Be an Alternative to Human Evaluations?David Cheng-Han Chiang, Hung-yi LeeACL 2023 · 254 次引用
  3. From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP ModelsShangbin Feng, Chan Young Park, Yuhan Liu, Yulia TsvetkovACL 2023 · 117 次引用
  4. Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software EngineeringRuiqi Wang, Jiyu Guo, Cuiyun Gao, Guodong Fan 等ISSTA 2025 · 27 次引用
  5. Measuring Political Bias in Large Language Models: What Is Said and How It Is SaidYejin Bang, Delong Chen, Nayeon Lee, Pascale FungACL 2024 · 21 次引用

相关 Paper

  1. Challenges and Opportunities in Information Manipulation Detection: An Examination of Wartime Russian MediaChan Young Park, Julia Mendelsohn, Anjalie Field, Yulia TsvetkovEMNLP Findings 2022 · 40 次引用
  2. Can AI-Generated Persuasion Be Detected? Persuaficial Benchmark and AI vs. Human Linguistic DifferencesArkadiusz Modzelewski, Pawel Golik, Anna Kolos, Giovanni Da San MartinoACL 2026 · 1 次引用
  3. NarratEX Dataset: Explaining the Dominant Narratives in News TextsNuno Guimarães, Purificação Silvano, Ricardo Campos, Alípio Mário Jorge 等EMNLP Findings 2025
  4. PolyNarrative: A Multilingual, Multilabel, Multi-domain Dataset for Narrative Extraction from News ArticlesNikolaos Nikolaidis, Nicolas Stefanovitch, Purificação Silvano, Dimitar Iliyanov Dimitrov 等ACL 2025
  5. EU DisinfoTest: a Benchmark for Evaluating Language Models' Ability to Detect Disinformation NarrativesWitold Sosnowski, Arkadiusz Modzelewski, Kinga Skorupska, Jahna Otterbacher 等EMNLP Findings 2024 · 3 次引用
黄昏的海面,两侧是细线勾勒的悬崖