Lune

EMNLP Findings2024

Do they mean 'us'? Interpreting Referring Expression variation under Intergroup Bias

Venkata S. Govindarajan, Matianyu Zang, Kyle Mahowald, David Beaver, Junyi Jessy Li

Abstract

The variations between in-group and out-group speech (intergroup bias) are subtle and could underlie many social phenomena like stereotype perpetuation and implicit bias. In this paper, we model intergroup bias as a tagging task on English sports comments from forums dedicated to fandom for NFL teams. We curate a dataset of over 6 million game-time comments from opposing perspectives (the teams in the game), each comment grounded in a non-linguistic description of the events that precipitated these comments (live win probabilities for each team). Expert and crowd annotations justify modeling the bias through tagging of implicit and explicit referring expressions and reveal the rich, contextual understanding of language and the world required for this task. For large-scale analysis of intergroup variation, we use LLMs for automated tagging, and discover that LLMs occasionally perform better when prompted with linguistic descriptions of the win probability at the time of the comment, rather than numerical probability. Further, large-scale tagging of comments using LLMs uncovers linear variations in the form of referent across win probabilities that distinguish in-group and out-group utterances.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Revisiting Relation Extraction in the era of Large Language Models cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on6

  1. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  2. Revisiting Relation Extraction in the era of Large Language ModelsSomin Wadhwa, Silvio Amir, Byron C. WallaceACL 2023 · 145 citations
  3. Towards Controllable Biases in Language GenerationEmily Sheng, Kai-Wei Chang, Prem Natarajan, Nanyun PengEMNLP Findings 2020 · 89 citations
  4. Social Bias Frames: Reasoning about Social and Power Implications of LanguageMaarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky et al.ACL 2020 · 16 citations
  5. Do "Undocumented Workers" == "Illegal Aliens"? Differentiating Denotation and Connotation in Vector SpacesAlbert Webson, Zhizhong Chen, Carsten Eickhoff, Ellie PavlickEMNLP 2020 · 8 citations

Related papers

  1. Who is better at math, Jenny or Jingzhen? Uncovering Stereotypes in Large Language ModelsZara Siddique, Liam D. Turner, Luis Espinosa AnkeEMNLP 2024 · 2 citations
  2. Relational AI: Facilitating Intergroup Cooperation with Socially Aware Conversational SupportElijah L. Claggett, Robert E. Kraut, Hirokazu ShiradoCHI 2025 · 10 citations
  3. From Fallback to Frontline: When Can LLMs be Superior Annotators of Human Perspectives?Hasan Amin, Harry Yizhou Tian, Xiaoni Duan, Chien-Ju Ho et al.ACL Findings 2026
  4. CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language ModelsNikita Nangia, Clara Vania, Rasika Bhalerao, Samuel R. BowmanEMNLP 2020 · 19 citations
  5. Are Stereotypes Leading LLMs' Zero-Shot Stance Detection ?Anthony Dubreuil, Antoine Gourru, Christine Largeron, Amine TrabelsiEMNLP 2025 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines