EMNLP Findings2023
Is a Prestigious Job the same as a Prestigious Country? A Case Study on Multilingual Sentence Embeddings and European Countries
Abstract
We study how multilingual sentence representations capture European countries and occupations and how this differs across European languages. We prompt the models with templated sentences that we machine-translate into 12 European languages and analyze the most prominent dimensions in the embeddings. Our analysis reveals that the most prominent feature in the embedding is the geopolitical distinction between Eastern and Western Europe and the country's economic strength in terms of GDP. When prompted specifically for job prestige, the embedding space clearly distinguishes high and low-prestige jobs. The occupational dimension is uncorrelated with the most dominant country dimensions in three out of four studied models. The exception is a small distilled model that exhibits a connection between occupational prestige and country of origin, which is a potential source of nationality-based discrimination. Our findings are consistent across languages.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages cites Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation
- Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages cites Unsupervised Cross-lingual Representation Learning at Scale
- Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation cites Unsupervised Cross-lingual Representation Learning at Scale
Cited by top-tier papers1
Builds on10
- MPNet: Masked and Permuted Pre-training for Language UnderstandingKaitao Song, Xu Tan, Tao Qin, Jianfeng Lu et al.NeurIPS 2020 · 1,957 citations
- Unsupervised Cross-lingual Representation Learning at ScaleAlexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary et al.ACL 2020 · 539 citations
- Language (Technology) is Power: A Critical Survey of "Bias" in NLPSu Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna M. WallachACL 2020 · 68 citations
- Making Monolingual Sentence Embeddings Multilingual using Knowledge DistillationNils Reimers, Iryna GurevychEMNLP 2020 · 54 citations
- Bitext Mining Using Distilled Sentence Representations for Low-Resource LanguagesKevin Heffernan, Onur Çelebi, Holger SchwenkEMNLP Findings 2022 · 47 citations
Related papers
- Improving Occupational ISCO Classification of Multilingual Swiss Job Postings with LLM-Refined Training DataAnn-Sophie Gnehm, Simon ClematideACL Findings 2025 · 1 citation
- Richer Countries and Richer RepresentationsKaitlyn Zhou, Kawin Ethayarajh, Dan JurafskyACL Findings 2022 · 8 citations
- On the Mutual Influence of Gender and Occupation in LLM RepresentationsHaozhe An, Connor Baumler, Abhilasha Sancheti, Rachel RudingerACL 2025
- French CrowS-Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than EnglishAurélie Névéol, Yoann Dupont, Julien Bezançon, Karën FortACL 2022 · 61 citations
- ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market DomainMike Zhang, Rob van der Goot, Barbara PlankACL 2023 · 14 citations
