Lune

ACL Findings2024

Real World Conversational Entity Linking Requires More Than Zero-Shots

Mohanna Hoveyda, Arjen P. de Vries, Faegheh Hasibi, Maarten de Rijke

Abstract

Entity linking (EL) in conversations faces notable challenges in practical applications, primarily due to the scarcity of entity-annotated conversational datasets and sparse knowledge bases (KB) containing domain-specific, longtail entities. We designed targeted evaluation scenarios to measure the efficacy of EL models under resource constraints. Our evaluation employs two KBs: Fandom, exemplifying realworld EL complexities, and the widely used Wikipedia. First, we assess EL models' ability to generalize to a new unfamiliar KB using Fandom and a novel zero-shot conversational entity linking dataset that we curated based on Reddit discussions on Fandom entities. We then evaluate the adaptability of EL models to conversational settings without prior training.

Our results indicate that current zero-shot EL models falter when introduced to new, domainspecific KBs without prior training, significantly dropping in performance. Our findings reveal that previous evaluation approaches fall short of capturing real-world complexities for zero-shot EL, highlighting the necessity for new approaches to design and assess conversational EL models to adapt to limited resources. The evaluation setup and the dataset proposed in this research are made publicly available. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Autoregressive Entity Retrieval cites Scalable Zero-shot Entity Linking with Dense Entity Retrieval
  • Entity-aware Transformers for Entity Search cites Autoregressive Entity Retrieval
  • Large Language Models Struggle to Learn Long-Tail Knowledge cites Autoregressive Entity Retrieval
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on6

  1. Large Language Models Struggle to Learn Long-Tail KnowledgeNikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace et al.ICML 2023 · 623 citations
  2. Scalable Zero-shot Entity Linking with Dense Entity RetrievalLedell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel et al.EMNLP 2020 · 336 citations
  3. Autoregressive Entity RetrievalNicola De Cao, Gautier Izacard, Sebastian Riedel, Fabio PetroniICLR 2021 · 200 citations
  4. Entity-aware Transformers for Entity SearchEmma J. Gerritse, Faegheh Hasibi, Arjen P. de VriesSIGIR 2022 · 27 citations
  5. BERT-ER: Query-specific BERT Entity Representations for Entity RankingShubham Chatterjee, Laura DietzSIGIR 2022 · 18 citations

Related papers

  1. Learn to Not Link: Exploring NIL Prediction in Entity LinkingFangwei Zhu, Jifan Yu, Hailong Jin, Lei Hou et al.ACL Findings 2023 · 6 citations
  2. Design Challenges in Low-resource Cross-lingual Entity LinkingXingyu Fu, Weijia Shi, Xiaodong Yu, Zian Zhao et al.EMNLP 2020 · 9 citations
  3. Entity Linking in 100 LanguagesJan A. Botha, Zifei Shan, Daniel GillickEMNLP 2020
  4. DESCGEN: A Distantly Supervised Datasetfor Generating Entity DescriptionsWeijia Shi, Mandar Joshi, Luke ZettlemoyerACL 2021
  5. WebIE: Faithful and Robust Information Extraction on the WebChenxi Whitehouse, Clara Vania, Alham Fikri Aji, Christos Christodoulopoulos et al.ACL 2023 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines