Annotating Dimensions of Social Perception in Text: A Sentence-Level Dataset of Warmth and Competence
Mutaz Ayesh, Saif M. Mohammad, Nedjma Ousidhoum
Abstract
Warmth (W) (often further broken down into Trust (T) and Sociability (S)) and Competence (C) are central dimensions along which people evaluate individuals and social groups (Fiske, 2018) . While these constructs are well established in social psychology, they are only starting to get attention in NLP research through word-level lexicons, which do not fully capture their contextual expression in larger text units and discourse. In this work, we introduce Warmth and Competence Sentences (W&C-Sent), the first sentence-level dataset annotated for warmth and competence. The dataset includes over 1,600 English sentence-target pairs annotated along three dimensions: trust and sociability (components of warmth), and competence 1 . The sentences in W&C-Sent are social media posts that express attitudes and opinions about specific individuals or social groups (the targets of our annotations). We describe the data collection, annotation, and quality-control procedures in detail, and evaluate a range of large language models (LLMs) on their ability to identify trust, sociability, and competence in text. W&C-Sent provides a new resource for analyzing warmth and competence in language and supports future research at the intersection of NLP and computational social science.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 11be2cf8-4774-4304-a5cb-2b81c8c7b6baBuilds on4
- Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language ModelsMyra Cheng, Esin Durmus, Dan JurafskyACL 2023 · 89 citations
- Language (Technology) is Power: A Critical Survey of "Bias" in NLPSu Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna M. WallachACL 2020 · 68 citations
- Probing Toxic Content in Large Pre-Trained Language ModelsNedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song et al.ACL 2021
- StereoSet: Measuring stereotypical bias in pretrained language modelsMoin Nadeem, Anna Bethke, Siva ReddyACL 2021
Related papers
- Words of Warmth: Trust and Sociability Norms for over 26k English WordsSaif M. MohammadACL 2025 · 5 citations
- Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET BenchmarkMinje Choi, Jiaxin Pei, Sagar Kumar, Chang Shu et al.EMNLP 2023 · 36 citations
- Ten Social Dimensions of Conversations and RelationshipsMinje Choi, Luca Maria Aiello, Krisztián Zsolt Varga, Daniele QuerciaWWW 2020 · 58 citations
- Social-Group-Agnostic Bias Mitigation via the Stereotype Content ModelAli Omrani, Alireza Salkhordeh Ziabari, Charles Yu, Preni Golazizian et al.ACL 2023 · 13 citations
- StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language ModelsSullam Jeoung, Yubin Ge, Jana DiesnerEMNLP 2023 · 3 citations
