Lune

EMNLP Findings2021

Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrieval

Yanmeng Wang, Jun Bai, Ye Wang, Jianfei Zhang, Wenge Rong, Zongcheng Ji, Shaojun Wang, Jing Xiao

Abstract

Dual-Encoders is a promising mechanism for answer retrieval in question answering (QA) systems. Currently most conventional Dual-Encoders learn the semantic representations of questions and answers merely through matching score. Researchers proposed to introduce the QA interaction features in scoring function but at the cost of low efficiency in inference stage. To keep independent encoding of questions and answers during inference stage, variational auto-encoder is further introduced to reconstruct answers (questions) from question (answer) embeddings as an auxiliary task to enhance QA interaction in representation learning in training stage. However, the needs of text generation and answer retrieval are different, which leads to hardness in training. In this work, we propose a framework to enhance the Dual-Encoders model with question answer cross-embeddings and a novel Geometry Alignment Mechanism (GAM) to align the geometry of embeddings from Dual-Encoders with that from Cross-Encoders. Extensive experimental results show that our framework significantly improves Dual-Encoders model and outperforms the state-of-the-art method on multiple answer retrieval datasets.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Dense Passage Retrieval for Open-Domain Question Answering cites Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Leveraging Estimated Transferability Over Human Intuition for Model Selection in Text RankingJun Bai, Zhuofan Chen, Zhenzi Li, Hanhua Hong et al.EMNLP 2024 · 2 citations

Builds on3

  1. Pre-training Tasks for Embedding-based Large-scale RetrievalWei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang et al.ICLR 2020 · 325 citations
  2. Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence ScoringSamuel Humeau, Kurt Shuster, Marie-Anne Lachaux, Jason WestonICLR 2020 · 316 citations
  3. Dense Passage Retrieval for Open-Domain Question AnsweringVladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis et al.EMNLP 2020 · 142 citations

Related papers

  1. Incomplete Cross-modal Retrieval with Dual-Aligned Variational AutoencodersMengmeng Jing, Jingjing Li, Lei Zhu, Ke Lu et al.ACM MM 2020 · 63 citations
  2. Pay More Attention to Relation Exploration for Knowledge Base Question AnsweringYong Cao, Xianzhi Li, Huiwen Liu, Wen Dai et al.ACL Findings 2023 · 11 citations
  3. Incomplete Multi-View Multi-label Learning via Disentangled Representation and Label Semantic EmbeddingXu Yan, Jun Yin, Jie WenCVPR 2025
  4. Cross-Modal Retrieval Augmentation for Multi-Modal ClassificationShir Gur, Natalia Neverova, Christopher Stauffer, Ser-Nam Lim et al.EMNLP Findings 2021 · 28 citations
  5. Revisit and Outstrip Entity Alignment: A Perspective of Generative ModelsLingbing Guo, Zhuo Chen, Jiaoyan Chen, Yin Fang et al.ICLR 2024 · 17 citations
Dusk over the sea between two cliffs drawn in fine vertical lines