Fairness Perceptions of Large Language Models

Large language models (LLMs) are increasingly 1 used for decision-making tasks where fairness is an 2 essential desideratum. But what does fairness even 3 mean to an LLM? To investigate this, we conduct 4 a comprehensive evaluation of how LLMs perceive 5 fairness in the context of resource allocation, using 6 both synthetic and real-world data. 7 We observe that various state-of-the-art LLMs, 8 when asked to be fair, prioritize improving col-9 lective welfare over distributing benefits equally. 10 Their perception of fairness is somewhat sensitive 11 to how user preferences are provided, but less so to 12 the real-world context of the decision-making task. 13 Finally, we show that the best strategy for align-14 ing an LLM's perception of fairness to a specific 15 criterion is to provide it as a mathematical objec-16 tive, without referencing "fairness", as this prevents 17 the LLM from mixing the given criterion with its 18 prior notions of fairness. Our results provide prac-19 tical insights regarding when to use LLMs for fair 20 decision-making and when using traditional algo-21 rithms may be more appropriate. 22 1 Introduction 23 The concept of fairness has captivated human thought for cen-24 turies, shaping the foundations of our core institutions, such 25 as democracy, law, and healthcare. But what does fairness 26 truly entail? While universally appealing, fairness is far from 27 universally defined, and its interpretation often depends on 28 the lens through which it is examined. 29 Fairness is a quintessential sociotechnical concept, ex-30 plored extensively across disciplines. Philosophy delib-31 erates the underlying principles of fairness, comparing 32 Rawls' [1971] egalitarianism to Harsanyi's [1975] utilitari-33 anism, and examining concepts such as desert, the right to 34 a minimum, and fair equality of opportunity. Meanwhile, 35 the machine learning literature takes a mathematical perspec-36 tive on fairness, and often narrows its focus to deal with 37 the most practically relevant issues such as mitigating race-38 or gender-based discrimination [Mehrabi et al., 2021]. The 39 fair division literature, at the intersection of economics and 40 1. What is fair in the eyes of LLMs? When LLMs are asked 2. What influences fairness perception? How does an 87 LLM's understanding of fairness depend on factors such 88 as the nature of agents and goods involved, and the fram-89 ing of the agents' preferences? 90 3. To what extent can we steer LLMs? Do the LLMs have 91 the reasoning abilities to optimize user-specified fairness 92 criteria? 93 Under the first two objectives, our goal is to identify pat-94 terns that are common across different LLMs. These patterns 95 may reflect perceptions of fairness encoded in the (largely 96 common) pretraining datasets that the LLMs are trained with 97 and, therefore, are likely to persist even as more capable 98 LLMs are deployed in the future. Under the third objective, 99 on the other hand, we seek to conduct an evaluation of the ca-100 pabilities of the current state-of-the-art (SOTA) LLMs. While 101 these models may soon be superseded, this portion of our 102 work contributes a framework that can be used for continuous 103 monitoring of the fairness capabilities of LLMs; thus, it con-104 tributes to the quickly-growing literature in AI on conducting 105 LLM evaluations on various dimensions such as safety, trust-106 worthiness, and inclination to hallucinate [Guo et al., 2023;