Lune

EMNLP Findings2023

A Word Sense Distribution-based approach for Semantic Change Prediction

Xiaohang Tang, Yi Zhou, Taichi Aida, Procheta Sen, Danushka Bollegala

Abstract

Semantic Change Detection (SCD) of words is an important task for various NLP applications that must make time-sensitive predictions. Some words are used over time in novel ways to express new meanings, and these new meanings establish themselves as novel senses of existing words. On the other hand, Word Sense Disambiguation (WSD) methods associate ambiguous words with sense ids, depending on the context in which they occur. Given this relationship between WSD and SCD, we explore the possibility of predicting whether a target word has its meaning changed between two corpora collected at different time steps, by comparing the distributions of senses of that word in each corpora. For this purpose, we use pretrained static sense embeddings to automatically annotate each occurrence of the target word in a corpus with a sense id. Next, we compute the distribution of sense ids of a target word in a given corpus. Finally, we use different divergence or distance measures to quantify the semantic change of the target word across the two given corpora. Our experimental results on SemEval 2020 Task 1 dataset show that word sense distributions can be accurately used to predict semantic changes of words in English, German, Swedish and Latin.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph Information cites SensEmBERT: Context-Enhanced Sense Embeddings for Multilingual Word Sense Disambiguation
  • With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense Disambiguation cites Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph Information
  • With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense Disambiguation cites SensEmBERT: Context-Enhanced Sense Embeddings for Multilingual Word Sense Disambiguation
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Knowledge-driven Augmentation and Retrieval for Integrative Temporal AdaptationWeisi Liu, Guangzeng Han, Xiaolei HuangACL 2026 · 1 citation
  2. HistLens: Mapping Idea Change across Concepts and CorporaYi Jing, Weiyun Qiu, Yihang Peng, Zhifang SuiACL 2026

Builds on8

  1. Mind the Gap: Assessing Temporal Generalization in Neural Language ModelsAngeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal et al.NeurIPS 2021 · 315 citations
  2. Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph InformationMichele Bevilacqua, Roberto NavigliACL 2020 · 145 citations
  3. SensEmBERT: Context-Enhanced Sense Embeddings for Multilingual Word Sense DisambiguationBianca Scarlini, Tommaso Pasini, Roberto NavigliAAAI 2020 · 121 citations
  4. Analysing Lexical Semantic Change with Contextualised Word RepresentationsMario Giulianelli, Marco Del Tredici, Raquel FernándezACL 2020 · 118 citations
  5. With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense DisambiguationBianca Scarlini, Tommaso Pasini, Roberto NavigliEMNLP 2020 · 95 citations

Related papers

  1. Swap and Predict - Predicting the Semantic Changes in Words across Corpora by Context SwappingTaichi Aida, Danushka BollegalaEMNLP Findings 2023 · 2 citations
  2. A Semantic Distance Metric Learning approach for Lexical Semantic Change DetectionTaichi Aida, Danushka BollegalaACL Findings 2024 · 4 citations
  3. Unsupervised Semantic Variation Prediction using the Distribution of Sibling EmbeddingsTaichi Aida, Danushka BollegalaACL Findings 2023 · 4 citations
  4. Fake it Till You Make it: Self-Supervised Semantic Shifts for Monolingual Word Embedding TasksMaurício Gruppi, Pin-Yu Chen, Sibel AdaliAAAI 2021 · 7 citations
  5. Sequential Modelling of the Evolution of Word Representations for Semantic Change DetectionAdam Tsakalidis, Maria LiakataEMNLP 2020 · 14 citations
Dusk over the sea between two cliffs drawn in fine vertical lines