Fair or Framed? Political Bias in News Articles Generated by LLMs
Junho Yoo, Youhyun Shin
Abstract
Despite biases in Large Language Models (LLMs) being widely researched, systematic explorations of political biases in news article generation tasks remain underexplored. This study evaluates political bias across seven LLMs by leveraging our PublicViews datasetextracted from the TwinViews-13k corpuscomprising 31 topics and 31,692 statements. We analyze 10,850 articles, finding left-leaning political bias persists in generation tasks, with neutral content remaining rare even under balanced opinion settings. Models exhibit asymmetric behavior in minority opinion scenarios, amplifying preferred viewpoints when in minority while conforming to majority opinions otherwise. Notably, all models employ "stanceflipping quotations" (altering supporters' statements to express opposite viewpoints) in 33-38% of quotations despite explicit instructions against distortion. Consistent with prior research, increased model size failed to enhance neutrality. This research measures political bias in LLM-generated news, analyzes its mechanisms, and reveals how opinion distribution and explicitness affect political bias expression. Our results highlight how LLMs can introduce unintended political bias in generative contexts. We publicly release our PublicViews corpus and code 1 .
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext be6f72c9-997c-4931-9f47-6b1e0f918d5cBuilds on12
- Aligning AI With Shared Human ValuesDan Hendrycks, Collin Burns, Steven Basart, Andrew Critch et al.ICLR 2021 · 878 citations
- Whose Opinions Do Language Models Reflect?Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee et al.ICML 2023 · 764 citations
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingMelanie Sclar, Yejin Choi, Yulia Tsvetkov, Alane SuhrICLR 2024 · 682 citations
- Evaluating the Moral Beliefs Encoded in LLMsNino Scherrer, Claudia Shi, Amir Feder, David M. BleiNeurIPS 2023 · 316 citations
- From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP ModelsShangbin Feng, Chan Young Park, Yuhan Liu, Yulia TsvetkovACL 2023 · 117 citations
Related papers
- When Bigger Isn't Better: A Comprehensive Fairness Evaluation of Political Bias in Multi-News SummarisationNannan Huang, Iffat Maab, Junichi YamagishiACL 2026
- Media Source Matters More Than Content: Unveiling Political Bias in LLM-Generated CitationsSunhao Dai, Zhanshuo Cao, Wenjie Wang, Liang Pang et al.EMNLP 2025
- Measuring and Mitigating Media Outlet Name Bias in Large Language ModelsSeong-Jin Park, Kang-Min KimEMNLP 2025
- Assessing Reliability and Political Bias In LLMs' Judgements of Formal and Material Inferences With Partisan ConclusionsReto Gubelmann, Ghassen KarrayACL 2025
- We Can Detect Your Bias: Predicting the Political Ideology of News ArticlesRamy Baly, Giovanni Da San Martino, James R. Glass, Preslav NakovEMNLP 2020 · 6 citations
