Lune

EMNLP Findings2022

GNN-encoder: Learning a Dual-encoder Architecture via Graph Neural Networks for Dense Passage Retrieval

Jiduan Liu, Jiahao Liu, Yang Yang, Jingang Wang, Wei Wu, Dongyan Zhao, Rui Yan

Abstract

Recently, retrieval models based on dense representations are dominant in passage retrieval tasks, due to their outstanding ability in terms of capturing semantics of input text compared to the traditional sparse vector space models. A common practice of dense retrieval models is to exploit a dual-encoder architecture to represent a query and a passage independently. Though efficient, such a structure loses interaction between the query-passage pair, resulting in inferior accuracy. To enhance the performance of dense retrieval models without loss of efficiency, we propose a GNN-encoder model in which query (passage) information is fused into passage (query) representations via graph neural networks that are constructed by queries and their top retrieved passages. By this means, we maintain a dual-encoder structure, and retain some interaction information between querypassage pairs in their representations, which enables us to achieve both efficiency and efficacy in passage retrieval. Evaluation results indicate that our method significantly outperforms the existing models on MSMARCO, Natural Questions and TriviaQA datasets, and achieves the new state-of-the-art on these datasets.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Dense Passage Retrieval for Open-Domain Question Answering cites ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
  • Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites Dense Passage Retrieval for Open-Domain Question Answering
  • Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval cites ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
  • RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking cites Dense Passage Retrieval for Open-Domain Question Answering
  • RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking cites ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Unify Graph Learning with Text: Unleashing LLM Potentials for Session SearchSonghao Wu, Quan Tu, Hong Liu, Jia Xu et al.WWW 2024 · 9 citations

Builds on12

  1. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text RetrievalLee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang et al.ICLR 2021 · 1,547 citations
  2. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTOmar Khattab, Matei ZahariaSIGIR 2020 · 1,246 citations
  3. BertGCN: Transductive Text Classification by Combining GNN and BERTYuxiao Lin, Yuxian Meng, Xiaofei Sun, Qinghong Han et al.ACL Findings 2021 · 229 citations
  4. RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingRuiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao et al.EMNLP 2021 · 147 citations
  5. Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis et al.EMNLP 2020 · 142 citations

Related papers

  1. PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage RetrievalRuiyang Ren, Shangwen Lv, Yingqi Qu, Jing Liu et al.ACL Findings 2021 · 59 citations
  2. KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question AnsweringDonghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu et al.ACL 2022 · 108 citations
  3. ConTextual Masked Auto-Encoder for Dense Passage RetrievalXing Wu, Guangyuan Ma, Meng Lin, Zijia Lin et al.AAAI 2023 · 34 citations
  4. Dense Hierarchical Retrieval for Open-domain Question AnsweringYe Liu, Kazuma Hashimoto, Yingbo Zhou, Semih Yavuz et al.EMNLP Findings 2021 · 24 citations
  5. Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware SamplingSebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin et al.SIGIR 2021 · 297 citations
Dusk over the sea between two cliffs drawn in fine vertical lines