Biased LLMs can Influence Political Decision-Making

As modern large language models (LLMs) become integral to everyday tasks, concerns about their inherent biases and their potential impact on human decision-making have emerged. While bias in models are well-documented, less is known about how these biases influence human decisions. This paper presents two interactive experiments investigating the effects of partisan bias in LLMs on political opinions and decision-making. Participants interacted freely with either a biased liberal, biased conservative, or unbiased control model while completing these tasks. We found that participants exposed to partisan biased models were significantly more likely to adopt opinions and make decisions which matched the LLM's bias. Even more surprising, this influence was seen when the model bias and personal political partisanship of the participant were opposite. However, we also discovered that prior knowledge of AI was weakly correlated with a reduction of the impact of the bias, highlighting the possible importance of AI education for robust mitigation of bias effects. Our findings not only highlight the critical effects of interacting with biased LLMs and its ability to impact public discourse and political conduct, but also highlights potential techniques for mitigating these risks in the future. Conservative Supported Topic Participant Partisanship Treatment Bias Beta Value t Value p-value Democrat Liberal -0.85 -2.38 0.02 Conservative 0.98 2.71 <0.01 Republican Liberal -0.79 -2.16 0.03 Conservative 0.19 0.55 0.58 Liberal Supported Topic Participant Partisanship Treatment Bias Beta Value t Value p-value Democrat Liberal 0.01 0.03 0.98 Conservative 1.44 3.82 <.01 Republican Liberal 0.20 0.58 0.56 Conservative 1.42 3.91 <.01 Table 1: Results of the Topic Opinion Task. All change in topic opinion ordinal logistic regression models were run without control variables. We ran two models, one for each participant partisanship. Bold indicates significant results with α = 0.05. Education Welfare Safety Veterans Education Welfare Safety Veterans Education Welfare Safety Veterans -10.