ACL Findings2023
Ambiguity Meets Uncertainty: Investigating Uncertainty Estimation for Word Sense Disambiguation
Abstract
Word sense disambiguation (WSD), which aims to determine an appropriate sense for a target word given its context, is crucial for natural language understanding. Existing supervised methods treat WSD as a classification task and have achieved remarkable performance. However, they ignore uncertainty estimation (UE) in the real-world setting, where the data is always noisy and out of distribution. This paper extensively studies UE on the benchmark designed for WSD. Specifically, we first compare four uncertainty scores for a state-of-the-art WSD model and verify that the conventional predictive probabilities obtained at the final layer of the model are inadequate to quantify uncertainty. Then, we examine the capability of capturing data and model uncertainties by the model with the selected UE score on well-designed test scenarios and discover that the model adequately reflects data uncertainty but underestimates model uncertainty. Furthermore, we explore numerous lexical properties that intrinsically affect data uncertainty and provide a detailed analysis of four critical aspects: the syntactic category, morphology, sense granularity, and semantic relations. The code is available at https://github.com/RyanLiut/WSD-UE.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- ConSeC: Word Sense Disambiguation as Continuous Sense Comprehension cites Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph Information
- ConSeC: Word Sense Disambiguation as Continuous Sense Comprehension cites Moving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encoders
- Nibbling at the Hard Core of Word Sense Disambiguation cites Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph Information
- Nibbling at the Hard Core of Word Sense Disambiguation cites ConSeC: Word Sense Disambiguation as Continuous Sense Comprehension
- Nibbling at the Hard Core of Word Sense Disambiguation cites Moving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encoders
Cited by top-tier papers1
Builds on10
- Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph InformationMichele Bevilacqua, Roberto NavigliACL 2020 · 145 citations
- ConSeC: Word Sense Disambiguation as Continuous Sense ComprehensionEdoardo Barba, Luigi Procopio, Roberto NavigliEMNLP 2021 · 60 citations
- Uncertainty Estimation of Transformer Predictions for Misclassification DetectionArtem Vazhentsev, Gleb Kuzmin, Artem Shelmanov, Akim Tsvigun et al.ACL 2022 · 59 citations
- Nibbling at the Hard Core of Word Sense DisambiguationMarco Maru, Simone Conia, Michele Bevilacqua, Roberto NavigliACL 2022 · 22 citations
- Moving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encodersTerra Blevins, Luke ZettlemoyerACL 2020 · 19 citations
Related papers
- SenseBERT: Driving Some Sense into BERTYoav Levine, Barak Lenz, Or Dagan, Ori Ram et al.ACL 2020 · 27 citations
- CSI: A Coarse Sense Inventory for 85% Word Sense DisambiguationCaterina Lacerra, Michele Bevilacqua, Tommaso Pasini, Roberto NavigliAAAI 2020 · 31 citations
- Towards General-Domain Word Sense Disambiguation: Distilling Large Language Model into Compact DisambiguatorLiqiang Ming, Sheng-hua Zhong, Yuncong LiEMNLP 2025
- A Synset Relation-enhanced Framework with a Try-again Mechanism for Word Sense DisambiguationMing Wang, Yinglin WangEMNLP 2020 · 29 citations
- FOOL ME IF YOU CAN! An Adversarial Dataset to Investigate the Robustness of LMs in Word Sense DisambiguationMohamad Ballout, Anne Dedert, Nohayr Abdelmoneim, Ulf Krumnack et al.EMNLP 2024 · 2 citations
