Lune

EMNLP Findings2025

Dynamic Injection of Entity Knowledge into Dense Retrievers

Ikuya Yamada, Ryokan Ri, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo

Abstract

Dense retrievers often struggle with queries involving less-frequent entities due to their limited entity knowledge. We propose the Knowledgeable Passage Retriever (KPR), a BERT-based retriever enhanced with a contextentity attention layer and dynamically updatable entity embeddings. This design enables KPR to incorporate external entity knowledge without retraining. Experiments on three datasets demonstrate that KPR consistently improves retrieval accuracy, with particularly large gains on the EntityQuestions dataset. When built on the off-the-shelf bgebase retriever, KPR achieves state-of-the-art performance among similarly sized models on two datasets. Models and code are released at github.com/knowledgeable-embedding/ knowledgeable-embedding.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Large Language Models Struggle to Learn Long-Tail Knowledge cites Retrieval Augmented Language Model Pre-Training
  • Retrieval Augmentation Reduces Hallucination in Conversation cites Retrieval Augmented Language Model Pre-Training
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on15

  1. Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
  2. Retrieval Augmented Language Model Pre-TrainingKelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat et al.ICML 2020 · 2,937 citations
  3. PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph CompilationJason Ansel, Edward Z. Yang, Horace He, Natalia Gimelshein et al.ASPLOS 2024 · 693 citations
  4. Large Language Models Struggle to Learn Long-Tail KnowledgeNikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace et al.ICML 2023 · 623 citations
  5. Retrieval Augmentation Reduces Hallucination in ConversationKurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela et al.EMNLP Findings 2021 · 579 citations

Related papers

  1. Entity-aware Transformers for Entity SearchEmma J. Gerritse, Faegheh Hasibi, Arjen P. de VriesSIGIR 2022 · 27 citations
  2. Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?Xilun Chen, Kushal Lakhotia, Barlas Oguz, Anchit Gupta et al.EMNLP Findings 2022 · 39 citations
  3. CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with TyposShengyao Zhuang, Guido ZucconSIGIR 2022 · 26 citations
  4. Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question AnsweringJialin Wu, Raymond J. MooneyEMNLP 2022 · 9 citations
  5. Knowledge-Graph Augmented Word Representations for Named Entity RecognitionQizhen He, Liang Wu, Yida Yin, Heming CaiAAAI 2020 · 30 citations
Dusk over the sea between two cliffs drawn in fine vertical lines