SCoUT: A Framework for Structured Stereotype Analysis in Language Models
Jinxuan Wu, Bin Li, Xiangyang Xue
Abstract
Existing stereotype auditing methods for Large Language Models (LLMs) typically rely on isolated rating schemes or task-specific probes, lacking theoretical grounding and failing to reveal internal organization beyond surface-level output patterns. In this paper, we introduce SCoUT (Stereotype Content-oriented Utility structure via Thurstonian modeling), a closed-loop framework that structurally models, explicitly probes, and functionally steers stereotype dimensions (warmth and competence) in LLMs. SCoUT first reconstructs a global stereotype utility structure aligned with Stereotype Content Model theory via Thurstonian comparative judgments. Across multiple open-source LLMs, this modeling achieves high pairwise-preference prediction accuracy (≥ 0.90 on larger-scale models) and exhibits strong cross-model consistency. Probing internal attention mechanisms localizes this structure to specific heads (Spearman’s ρ up to 0.83 for warmth and 0.90 for competence) and surfaces a salient asymmetry between warmth and competence. Further, targeted inference-time activation modifications on these dimension-sensitive heads consistently steer model outputs along the intended axes. By bridging behavioral measurement with internal representation and controllable steering, SCoUT offers an end-to-end framework that uncovers and interprets the latent structure of stereotypes, advancing stereotype auditing from surface detection to structural analysis.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 3996a279-9acc-42ae-bab4-b87cdc58cfb8Builds on5
- Inference-Time Intervention: Eliciting Truthful Answers from a Language ModelKenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister et al.NeurIPS 2023 · 1,549 citations
- CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language ModelsNikita Nangia, Clara Vania, Rasika Bhalerao, Samuel R. BowmanEMNLP 2020 · 19 citations
- StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language ModelsSullam Jeoung, Yubin Ge, Jana DiesnerEMNLP 2023 · 3 citations
- Linear Representations of Political Perspective Emerge in Large Language ModelsJunsol Kim, James Evans, Aaron ScheinICLR 2025
- StereoSet: Measuring stereotypical bias in pretrained language modelsMoin Nadeem, Anna Bethke, Siva ReddyACL 2021
Related papers
- Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content ModelKathleen C. Fraser, Isar Nejadgholi, Svetlana KiritchenkoACL 2021
- Social-Group-Agnostic Bias Mitigation via the Stereotype Content ModelAli Omrani, Alireza Salkhordeh Ziabari, Charles Yu, Preni Golazizian et al.ACL 2023 · 13 citations
- Annotating Dimensions of Social Perception in Text: A Sentence-Level Dataset of Warmth and CompetenceMutaz Ayesh, Saif M. Mohammad, Nedjma OusidhoumACL 2026
- Uncovering Competency Gaps in Large Language Models and Their BenchmarksMaty Bohacek, Nino Scherrer, Nicholas Dufour, Thomas Leung et al.ICML 2026
- The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM ObjectivesMatthieu Bou, Nyal Patel, Arjun Jagota, Satyapriya Krishna et al.ICLR 2026 · 1 citation
