Lune

ACL Findings2023

Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems

Chiyu Song, Hongliang He, Haofei Yu, Pengfei Fang, Leyang Cui, Zhenzhong Lan

摘要

Sample-and-rank is a key decoding strategy for modern generation-based dialogue systems. It helps achieve diverse and high-quality responses by selecting an answer from a small pool of generated candidates. The current stateof-the-art ranking methods mainly use an encoding paradigm called Cross-Encoder, which separately encodes each context-candidate pair and ranks the candidates according to their fitness scores. However, Cross-Encoder repeatedly encodes the same lengthy context for each candidate, resulting in high computational costs. Poly-Encoder addresses the above problems by reducing the interaction between context and candidates, but with a price of performance drop. In this work, we develop a new paradigm called Uni-Encoder 1 , that keeps the full attention over each pair as in Cross-Encoder while only encoding the context once, as in Poly-Encoder. Uni-Encoder encodes all the candidates with the context in one forward pass. We use the same positional embedding for all candidates to ensure they are treated equally and design a new attention mechanism to avoid confusion. Our Uni-Encoder can simulate other ranking paradigms using different attention and response concatenation methods. Extensive experiments show that our proposed paradigm achieves new state-of-theart results on four benchmark datasets with high computational efficiency. For instance, it improves R 10 @1 by 2.9% with an approximately 4× faster inference speed on the Ubuntu V2 dataset.

问 Lune

几分钟读透这篇 Paper,不用再花一下午。

Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。

可以从这些问题问起

Lune 可以查阅

全文 · 相关的 Paper

获取答案

免费开始,无需绑卡

脉络

  • Chain of Hindsight aligns Language Models with Feedback 引用了 Learning to summarize with human feedback
  • Chain of Hindsight aligns Language Models with Feedback 引用了 Training language models to follow instructions with human feedback
它引用的引用它的从左到右按年份排列,引用越多,位置越高。

它引用的 Paper9

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida 等NeurIPS 2022 · 24,707 次引用
  2. Chain of Hindsight aligns Language Models with FeedbackHao Liu, Carmelo Sferrazza, Pieter AbbeelICLR 2024 · 162 次引用
  3. Learning to summarize with human feedbackNisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler 等NeurIPS 2020 · 124 次引用
  4. Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based DialoguesRuijian Xu, Chongyang Tao, Daxin Jiang, Xueliang Zhao 等AAAI 2021 · 76 次引用
  5. Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response SelectionTaesun Whang, Dongyub Lee, Dongsuk Oh, Chanhee Lee 等AAAI 2021 · 70 次引用

相关 Paper

  1. ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking InferenceKai Hui, Honglei Zhuang, Tao Chen, Zhen Qin 等ACL Findings 2022 · 5 次引用
  2. Language Ranker: A Lightweight Ranking framework for LLM DecodingChenheng Zhang, Tianqi Du, Jizhe Zhang, Mingqing Xiao 等NeurIPS 2025 · 3 次引用
  3. Comparing Neighbors Together Makes it Easy: Jointly Comparing Multiple Candidates for Efficient and Effective RetrievalJonghyun Song, Cheyon Jin, Wenlong Zhao, Andrew McCallum 等EMNLP 2024 · 3 次引用
  4. RankGen: Improving Text Generation with Large Ranking ModelsKalpesh Krishna, Yapei Chang, John Wieting, Mohit IyyerEMNLP 2022 · 27 次引用
  5. Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language ModelsQi Liu, Bo Wang, Nan Wang, Jiaxin MaoWWW 2025 · 26 次引用
黄昏的海面,两侧是细线勾勒的悬崖