Lune

EMNLP2025Top-tier venue

'Rich Dad, Poor Lad': How do Large Language Models Contextualize Socioeconomic Factors in College Admission ?

Huy Nghiem, Phuong-Anh Nguyen-Le, John Prindle, Rachel Rudinger, Hal Daumé III

Abstract

Large Language Models (LLMs) are increasingly involved in high-stakes domains, yet how they reason about socially-sensitive decisions still remains underexplored. We present a largescale audit of LLMs' treatment of socioeconomic status (SES) in college admissions decisions using a novel dual-process framework inspired by cognitive science. Leveraging a synthetic dataset of 30,000 applicant profiles 1 grounded in real-world correlations, we prompt 4 open-source LLMs (Qwen 2, Mistral v0.3, Gemma 2, Llama 3.1) under 2 modes: a fast, decision-only setup (System 1) and a slower, explanation-based setup (System 2). Results from 5 million prompts reveals that LLMs consistently favor low-SES applicants-even when controlling for academic performance-and that System 2 amplifies this tendency by explicitly invoking SES as compensatory justification, highlighting both their potential and volatility as decision-makers. We then propose DPAF, a dual-process audit framework to probe LLMs' reasoning behaviors in sensitive applications.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Fast Model DeBias with Machine Unlearning cites Post-processing for Individual Fairness
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on12

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought PromptingMiles Turpin, Julian Michael, Ethan Perez, Samuel R. BowmanNeurIPS 2023 · 1,792 citations
  3. Safe RLHF: Safe Reinforcement Learning from Human FeedbackJosef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji et al.ICLR 2024 · 656 citations
  4. Post-processing for Individual FairnessFelix Petersen, Debarghya Mukherjee, Yuekai Sun, Mikhail YurochkinNeurIPS 2021 · 115 citations
  5. Fast Model DeBias with Machine UnlearningRuizhe Chen, Jianfei Yang, Huimin Xiong, Jianhong Bai et al.NeurIPS 2023 · 110 citations

Related papers

  1. Take Out Your Calculators: Estimating the Real Difficulty of Question Items with LLM Student SimulationsChristabel Acquaye, Yi Ting Huang, Marine Carpuat, Rachel RudingerACL Findings 2026 · 2 citations
  2. Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning TasksFangru Lin, Shaoguang Mao, Emanuele La Malfa, Valentin Hofmann et al.ACL 2025 · 14 citations
  3. Biases in the Blind Spot: Detecting What LLMs Fail to MentionIván Arcuschin, David Chanin, Adrià Garriga-Alonso, Oana-Maria CamburuICML 2026 · 7 citations
  4. To Mask or to Mirror: Human-AI Alignment in Collective ReasoningCrystal Qian, Aaron T. Parisi, Clémentine Bouleau, Vivian Tsai et al.EMNLP 2025 · 1 citation
  5. Prompting Fairness: Integrating Causality to Debias Large Language ModelsJingling Li, Zeyu Tang, Xiaoyu Liu, Peter Spirtes et al.ICLR 2025
Dusk over the sea between two cliffs drawn in fine vertical lines