Lune

AAAI2026Top-tier venue

"As Eastern Powers, I Will Veto.": An Investigation of Nation-Level Bias of Large Language Models in International Relations

Jonghyeon Choi, Yeonjun Choi, Hyun-chul Kim, Beakcheol Jang

Abstract

This paper provides an early effort to systematically examine nation-level biases exhibited by Large Language Models (LLMs) within the domain of International Relations (IR), a dimension that has remained largely unexplored in prior research. Leveraging historical records from the United Nations Security Council (UNSC), we developed a bias evaluation framework comprising three distinct tests to explore nation-level bias in various LLMs, with a particular focus on the five permanent members of the UNSC. Experimental results show that, even with the general bias patterns across models (e.g., favorable biases toward the western nations, and unfavorable biases toward Russia), these still vary based on the LLM. Notably, even within the same LLM, the direction and magnitude of bias for a nation change depending on the evaluation context. This observation suggests that LLM biases are fundamentally multidimensional, varying across models and tasks. We also observe that models with stronger reasoning abilities show reduced bias and better prediction performance. Building on this finding, we introduce a debiasing framework that improves LLMs’ factual reasoning combining Retrieval-Augmented Generation with Reflexion-based self-reflection techniques. Experiments show it effectively reduces nation-level bias, and improves performance, particularly in GPT-4o-mini and LLama-3.3-70B. Our findings emphasize the need to assess nation-level bias alongside prediction performance when applying LLMs in the IR domain.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on15

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  2. Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
  3. Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMsShashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan et al.ICLR 2024 · 212 citations
  4. BBQ: A hand-built bias benchmark for question answeringAlicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar et al.ACL Findings 2022 · 186 citations
  5. Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language ModelsMosh Levy, Alon Jacoby, Yoav GoldbergACL 2024 · 77 citations

Related papers

  1. Are LLMs Rational Investors? A Study on the Financial Bias in LLMsYuhang Zhou, Yuchen Ni, Zhiheng Xi, Zhangyue Yin et al.ACL Findings 2025 · 3 citations
  2. Pride and Prejudice: LLM Amplifies Self-Bias in Self-RefinementWenda Xu, Guanglei Zhu, Xuandong Zhao, Liangming Pan et al.ACL 2024
  3. Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-ReflectionYachao Zhao, Bo Wang, Yan Wang, Dongming Zhao et al.ACL Findings 2025 · 4 citations
  4. UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN ManipulationHanzhang Zhou, Zijian Feng, Zixiao Zhu, Junlang Qian et al.NeurIPS 2024 · 43 citations
  5. Persona-Assigned Large Language Models Exhibit Human-Like Motivated ReasoningSaloni Dash, Amélie Reymond, Emma S. Spiro, Aylin CaliskanACL Findings 2026 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines