Bias in the Mirror : Are LLMs opinions robust to their own adversarial attacks
Virgile Rennard, Christos Xypolopoulos, Michalis Vazirgiannis
Abstract
Large language models (LLMs) inherit biases from their training data and alignment processes, influencing their responses in subtle ways. While many studies have examined these biases, little work has explored their robustness during interactions. In this paper, we introduce a novel approach where two instances of an LLM engage in self-debate, arguing opposing viewpoints to persuade a neutral version of the model. Through this, we evaluate how firmly biases hold and whether models are susceptible to reinforcing misinformation or shifting to harmful viewpoints. Our experiments span multiple LLMs of varying sizes, origins, and languages, providing deeper insights into bias persistence and flexibility across linguistic and cultural contexts.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 54b30282-6546-4c2b-a2ae-d10ed265be86Builds on7
- Whose Opinions Do Language Models Reflect?Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee et al.ICML 2023 · 764 citations
- Debating with More Persuasive LLMs Leads to More Truthful AnswersAkbir Khan, John Hughes, Dan Valentine, Laura Ruis et al.ICML 2024 · 244 citations
- From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP ModelsShangbin Feng, Chan Young Park, Yuhan Liu, Yulia TsvetkovACL 2023 · 117 citations
- Systematic Biases in LLM Simulations of DebatesAmir Taubenfeld, Yaniv Dover, Roi Reichart, Ariel GoldsteinEMNLP 2024 · 35 citations
- Measuring Political Bias in Large Language Models: What Is Said and How It Is SaidYejin Bang, Delong Chen, Nayeon Lee, Pascale FungACL 2024 · 21 citations
Related papers
- Inertia in Moral and Value Judgments of Large Language ModelsBruce W. Lee, Yeongheon Lee, Hyunsoo ChoACL 2026 · 5 citations
- The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive ConversationRongwu Xu, Brian S. Lin, Shujian Yang, Tianqi Zhang et al.ACL 2024
- Stimulate the Critical Thinking of LLMs via Debiasing DiscussionRuiyu Xiao, Lei Wu, Yuanxing Liu, Weinan Zhang et al.EMNLP 2025 · 1 citation
- Breaking Mental Set to Improve Reasoning through Diverse Multi-Agent DebateYexiang Liu, Jie Cao, Zekun Li, Ran He et al.ICLR 2025
- From Single to Societal: Analyzing Persona-Induced Bias in Multi-Agent InteractionsJiayi Li, Xiao Liu, Yansong FengAAAI 2026 · 3 citations
