Lune

ACL Findings2023

Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems

Chiyu Song, Hongliang He, Haofei Yu, Pengfei Fang, Leyang Cui, Zhenzhong Lan

Abstract

Sample-and-rank is a key decoding strategy for modern generation-based dialogue systems. It helps achieve diverse and high-quality responses by selecting an answer from a small pool of generated candidates. The current stateof-the-art ranking methods mainly use an encoding paradigm called Cross-Encoder, which separately encodes each context-candidate pair and ranks the candidates according to their fitness scores. However, Cross-Encoder repeatedly encodes the same lengthy context for each candidate, resulting in high computational costs. Poly-Encoder addresses the above problems by reducing the interaction between context and candidates, but with a price of performance drop. In this work, we develop a new paradigm called Uni-Encoder 1 , that keeps the full attention over each pair as in Cross-Encoder while only encoding the context once, as in Poly-Encoder. Uni-Encoder encodes all the candidates with the context in one forward pass. We use the same positional embedding for all candidates to ensure they are treated equally and design a new attention mechanism to avoid confusion. Our Uni-Encoder can simulate other ranking paradigms using different attention and response concatenation methods. Extensive experiments show that our proposed paradigm achieves new state-of-theart results on four benchmark datasets with high computational efficiency. For instance, it improves R 10 @1 by 2.9% with an approximately 4× faster inference speed on the Ubuntu V2 dataset.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Chain of Hindsight aligns Language Models with Feedback cites Learning to summarize with human feedback
  • Chain of Hindsight aligns Language Models with Feedback cites Training language models to follow instructions with human feedback
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on9

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. Chain of Hindsight aligns Language Models with FeedbackHao Liu, Carmelo Sferrazza, Pieter AbbeelICLR 2024 · 162 citations
  3. Learning to summarize with human feedbackNisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler et al.NeurIPS 2020 · 124 citations
  4. Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based DialoguesRuijian Xu, Chongyang Tao, Daxin Jiang, Xueliang Zhao et al.AAAI 2021 · 76 citations
  5. Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response SelectionTaesun Whang, Dongyub Lee, Dongsuk Oh, Chanhee Lee et al.AAAI 2021 · 70 citations

Related papers

  1. ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking InferenceKai Hui, Honglei Zhuang, Tao Chen, Zhen Qin et al.ACL Findings 2022 · 5 citations
  2. Language Ranker: A Lightweight Ranking framework for LLM DecodingChenheng Zhang, Tianqi Du, Jizhe Zhang, Mingqing Xiao et al.NeurIPS 2025 · 3 citations
  3. Comparing Neighbors Together Makes it Easy: Jointly Comparing Multiple Candidates for Efficient and Effective RetrievalJonghyun Song, Cheyon Jin, Wenlong Zhao, Andrew McCallum et al.EMNLP 2024 · 3 citations
  4. RankGen: Improving Text Generation with Large Ranking ModelsKalpesh Krishna, Yapei Chang, John Wieting, Mohit IyyerEMNLP 2022 · 27 citations
  5. Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language ModelsQi Liu, Bo Wang, Nan Wang, Jiaxin MaoWWW 2025 · 26 citations
Dusk over the sea between two cliffs drawn in fine vertical lines