Annotating Dimensions of Social Perception in Text: A Sentence-Level Dataset of Warmth and Competence
Mutaz Ayesh, Saif M. Mohammad, Nedjma Ousidhoum
摘要
Warmth (W) (often further broken down into Trust (T) and Sociability (S)) and Competence (C) are central dimensions along which people evaluate individuals and social groups (Fiske, 2018) . While these constructs are well established in social psychology, they are only starting to get attention in NLP research through word-level lexicons, which do not fully capture their contextual expression in larger text units and discourse. In this work, we introduce Warmth and Competence Sentences (W&C-Sent), the first sentence-level dataset annotated for warmth and competence. The dataset includes over 1,600 English sentence-target pairs annotated along three dimensions: trust and sociability (components of warmth), and competence 1 . The sentences in W&C-Sent are social media posts that express attitudes and opinions about specific individuals or social groups (the targets of our annotations). We describe the data collection, annotation, and quality-control procedures in detail, and evaluate a range of large language models (LLMs) on their ability to identify trust, sociability, and competence in text. W&C-Sent provides a new resource for analyzing warmth and competence in language and supports future research at the intersection of NLP and computational social science.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper4
- Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language ModelsMyra Cheng, Esin Durmus, Dan JurafskyACL 2023 · 被引用 89 次
- Language (Technology) is Power: A Critical Survey of "Bias" in NLPSu Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna M. WallachACL 2020 · 被引用 68 次
- Probing Toxic Content in Large Pre-Trained Language ModelsNedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song 等ACL 2021
- StereoSet: Measuring stereotypical bias in pretrained language modelsMoin Nadeem, Anna Bethke, Siva ReddyACL 2021
相关 Paper
- Words of Warmth: Trust and Sociability Norms for over 26k English WordsSaif M. MohammadACL 2025 · 被引用 5 次
- Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET BenchmarkMinje Choi, Jiaxin Pei, Sagar Kumar, Chang Shu 等EMNLP 2023 · 被引用 36 次
- Ten Social Dimensions of Conversations and RelationshipsMinje Choi, Luca Maria Aiello, Krisztián Zsolt Varga, Daniele QuerciaWWW 2020 · 被引用 58 次
- Social-Group-Agnostic Bias Mitigation via the Stereotype Content ModelAli Omrani, Alireza Salkhordeh Ziabari, Charles Yu, Preni Golazizian 等ACL 2023 · 被引用 13 次
- StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language ModelsSullam Jeoung, Yubin Ge, Jana DiesnerEMNLP 2023 · 被引用 3 次
