ACL Findings2026
Auditing LLM Responses to Harmful Stereotypes Targeting Mental Health Groups
Arka Dutta, Rijul Magu, Sean Kim, Seohee Yoon, Munmun De Choudhury, Ashiqur R. KhudaBukhsh
摘要
Large Language Models (LLMs) can exhibit imbalanced biases against vulnerable groups, but how they rationalize stereotypes and rights restrictions targeting mental health entities remains underexplored. We audit a broad suite of open-weight LLMs on stereotype-justification prompts tied to mental health identities. We find that several widely used models endorse harmful stereotypes when explicitly asked to justify them, with endorsement varying across model families, versions, and mental health conditions. We also show that widely used harmful-content evaluation and moderation frameworks often miss these nuanced, discriminatory responses, highlighting a gap in current AI safety evaluation for mental health groups.
问 Lune
尽管问,回答会引用原文。
Lune 会读完全文和相关的 Paper,每个回答都附上所依据的原文段落。
它引用的 Paper7
- Whose Opinions Do Language Models Reflect?Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee 等ICML 2023 · 764 次引用
- Synthetic Lies: Understanding AI-Generated Misinformation and Evaluating Algorithmic and Human SolutionsJiawei Zhou, Yixuan Zhang, Qianni Luo, Andrea G. Parker 等CHI 2023 · 283 次引用
- "Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference LettersYixin Wan, George Pu, Jiao Sun, Aparna Garimella 等EMNLP Findings 2023 · 139 次引用
- Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare QueriesYiqiao Jin, Mohit Chandra, Gaurav Verma, Yibo Hu 等WWW 2024 · 126 次引用
- Language (Technology) is Power: A Critical Survey of "Bias" in NLPSu Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna M. WallachACL 2020 · 68 次引用
相关 Paper
- LLMs Reproduce Stereotypes of Sexual and Gender MinoritiesRuby Ostrow, Adam LopezEMNLP Findings 2025 · 7 次引用
- Rethinking Research on Stereotypes: An Analysis through Social Psychological and Computational PerspectivesKaustubh Shivshankar Shejole, Pushpak BhattacharyyaACL Findings 2026
- Gendered Mental Health Stigma in Masked Language ModelsInna W. Lin, Lucille Njoo, Anjalie Field, Ashish Sharma 等EMNLP 2022 · 15 次引用
- AccessEval: Benchmarking Disability Bias in Large Language ModelsSrikant Panda, Amit Agarwal, Hitesh Laxmichand PatelEMNLP 2025 · 2 次引用
- A Conditional Companion: Lived Experiences of People with Mental Health Disorders Using LLMs: Conditional Companion: LLMs & Mental HealthAditya Kumar Purohit, Hendrik HeuerCHI 2026 · 3 次引用
