ACL Findings2023
Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems
Chiyu Song, Hongliang He, Haofei Yu, Pengfei Fang, Leyang Cui, Zhenzhong Lan
摘要
Sample-and-rank is a key decoding strategy for modern generation-based dialogue systems. It helps achieve diverse and high-quality responses by selecting an answer from a small pool of generated candidates. The current stateof-the-art ranking methods mainly use an encoding paradigm called Cross-Encoder, which separately encodes each context-candidate pair and ranks the candidates according to their fitness scores. However, Cross-Encoder repeatedly encodes the same lengthy context for each candidate, resulting in high computational costs. Poly-Encoder addresses the above problems by reducing the interaction between context and candidates, but with a price of performance drop. In this work, we develop a new paradigm called Uni-Encoder 1 , that keeps the full attention over each pair as in Cross-Encoder while only encoding the context once, as in Poly-Encoder. Uni-Encoder encodes all the candidates with the context in one forward pass. We use the same positional embedding for all candidates to ensure they are treated equally and design a new attention mechanism to avoid confusion. Our Uni-Encoder can simulate other ranking paradigms using different attention and response concatenation methods. Extensive experiments show that our proposed paradigm achieves new state-of-theart results on four benchmark datasets with high computational efficiency. For instance, it improves R 10 @1 by 2.9% with an approximately 4× faster inference speed on the Ubuntu V2 dataset.
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
脉络
它引用的 Paper9
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida 等NeurIPS 2022 · 24,707 次引用
- Chain of Hindsight aligns Language Models with FeedbackHao Liu, Carmelo Sferrazza, Pieter AbbeelICLR 2024 · 162 次引用
- Learning to summarize with human feedbackNisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler 等NeurIPS 2020 · 124 次引用
- Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based DialoguesRuijian Xu, Chongyang Tao, Daxin Jiang, Xueliang Zhao 等AAAI 2021 · 76 次引用
- Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response SelectionTaesun Whang, Dongyub Lee, Dongsuk Oh, Chanhee Lee 等AAAI 2021 · 70 次引用
相关 Paper
- ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking InferenceKai Hui, Honglei Zhuang, Tao Chen, Zhen Qin 等ACL Findings 2022 · 5 次引用
- Language Ranker: A Lightweight Ranking framework for LLM DecodingChenheng Zhang, Tianqi Du, Jizhe Zhang, Mingqing Xiao 等NeurIPS 2025 · 3 次引用
- Comparing Neighbors Together Makes it Easy: Jointly Comparing Multiple Candidates for Efficient and Effective RetrievalJonghyun Song, Cheyon Jin, Wenlong Zhao, Andrew McCallum 等EMNLP 2024 · 3 次引用
- RankGen: Improving Text Generation with Large Ranking ModelsKalpesh Krishna, Yapei Chang, John Wieting, Mohit IyyerEMNLP 2022 · 27 次引用
- Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language ModelsQi Liu, Bo Wang, Nan Wang, Jiaxin MaoWWW 2025 · 26 次引用
