ACL Findings2022
Richer Countries and Richer Representations
Kaitlyn Zhou, Kawin Ethayarajh, Dan Jurafsky
Abstract
We examine whether some countries are more richly represented in embedding space than others. We find that countries whose names occur with low frequency in training corpora are more likely to be tokenized into subwords, are less semantically distinct in embedding space, and are less likely to be correctly predicted: e.g., Ghana (the correct answer and invocabulary) is not predicted for, "The country producing the most cocoa is [MASK].". Although these performance discrepancies and representational harms are due to frequency, we find that frequency is highly correlated with a country's GDP; thus perpetuating historic power and wealth inequalities. We analyze the effectiveness of mitigation strategies; recommend that researchers report training word frequencies; and recommend future work for the community to define and design representational guarantees.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
5 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Language (Technology) is Power: A Critical Survey of "Bias" in NLP cites Adhering, Steering, and Queering: Treatment of Gender in Natural Language Generation
- RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models cites Language (Technology) is Power: A Critical Survey of "Bias" in NLP
- Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models cites Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings
Cited by top-tier papers5
- Hate Personified: Investigating the role of LLMs in content moderationSarah Masud, Sahajpreet Singh, Viktor Hangya, Alexander Fraser et al.EMNLP 2024 · 6 citations
- Do You Know About My Nation? Investigating Multilingual Language Models' Cultural Literacy Through Factual KnowledgeEshaan Tanwar, Anwoy Chatterjee, Michael Saxon, Alon Albalak et al.EMNLP 2025 · 4 citations
- Can language models learn analogical reasoning? Investigating training objectives and comparisons to human performanceMolly R. Petersen, Lonneke van der PlasEMNLP 2023 · 3 citations
- Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal GesturesAkhila Yerukola, Saadia Gabriel, Nanyun Peng, Maarten SapACL 2025 · 3 citations
- AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data FiltersLi Lucy, Suchin Gururangan, Luca Soldaini, Emma Strubell et al.ACL 2024 · 2 citations
Builds on6
- RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language ModelsSamuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi et al.EMNLP Findings 2020 · 692 citations
- Interpreting Pretrained Contextualized Representations via Reductions to Static EmbeddingsRishi Bommasani, Kelly Davis, Claire CardieACL 2020 · 137 citations
- Language (Technology) is Power: A Critical Survey of "Bias" in NLPSu Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna M. WallachACL 2020 · 68 citations
- Adhering, Steering, and Queering: Treatment of Gender in Natural Language GenerationYolande A. A. Strengers, Lizhen Qu, Qiongkai Xu, Jarrod KnibbeCHI 2020 · 31 citations
- Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language ModelsRobert Wolfe, Aylin CaliskanEMNLP 2021 · 27 citations
Related papers
- Zipfian WhiteningSho Yokoi, Han Bao, Hiroto Kurita, Hidetoshi ShimodairaNeurIPS 2024 · 3 citations
- Uncovering Currency Bias and Syntax Gap in Text Embedding ModelsSaurav Sudevan, Harsh Pal, Yatin Katyal, Sahil ManchandaACL Findings 2026
- Geographical Erasure in Language GenerationPola Schwöbel, Jacek Golebiowski, Michele Donini, Cédric Archambeau et al.EMNLP Findings 2023 · 3 citations
- Is anisotropy really the cause of BERT embeddings not being semantic?Alejandro Fuster Baggetto, Víctor FresnoEMNLP Findings 2022 · 9 citations
- Is a Prestigious Job the same as a Prestigious Country? A Case Study on Multilingual Sentence Embeddings and European CountriesJindrich LibovickýEMNLP Findings 2023
