DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection
Yongkang Liu, Shi Feng, Wei Gao, Daling Wang, Yifei Zhang
Abstract
Current end-to-end retrieval-based dialogue systems are mainly based on Recurrent Neural Networks or Transformers with attention mechanisms. Although promising results have been achieved, these models often suffer from slow inference or huge number of parameters. In this paper, we propose a novel lightweight fully convolutional architecture, called DialogConv, for response selection. DialogConv is exclusively built on top of convolution to extract matching features of context and response. Dialogues are modeled in 3D views, where DialogConv performs convolution operations on embedding view, word view and utterance view to capture richer semantic information from multiple contextual views. On the four benchmark datasets, compared with state-of-the-art baselines, Di-alogConv is on average about 8.5× smaller in size, and 79.39× and 10.64× faster on CPU and GPU devices, respectively. At the same time, DialogConv achieves the competitive effectiveness of response selection.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on6
- ALBERT: A Lite BERT for Self-supervised Learning of Language RepresentationsZhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel et al.ICLR 2020 · 7,418 citations
- CoAtNet: Marrying Convolution and Attention for All Data SizesZihang Dai, Hanxiao Liu, Quoc V. Le, Mingxing TanNeurIPS 2021 · 1,747 citations
- Lite Transformer with Long-Short Range AttentionZhanghao Wu, Zhijian Liu, Ji Lin, Yujun Lin et al.ICLR 2020 · 379 citations
- MuTual: A Dataset for Multi-Turn Dialogue ReasoningLeyang Cui, Yu Wu, Shujie Liu, Yue Zhang et al.ACL 2020 · 115 citations
- A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-trainingYongkang Liu, Shi Feng, Daling Wang, Kaisong Song et al.AAAI 2021 · 23 citations
Related papers
- Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain ConversationsWei Chen, Yeyun Gong, Can Xu, Huang Hu et al.ACL 2022
- Towards Efficient Dialogue Pre-training with Transferable and Interpretable Latent StructureXueliang Zhao, Lemao Liu, Tingchen Fu, Shuming Shi et al.EMNLP 2022 · 3 citations
- A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded ConversationsChongyang Tao, Changyu Chen, Jiazhan Feng, Ji-Rong Wen et al.ACL 2021
- DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank UtterancesXiaodong Gu, Kang Min Yoo, Jung-Woo HaAAAI 2021 · 83 citations
- Generating Dialogue Responses from a Semantic Latent SpaceWei-Jen Ko, Avik Ray, Yilin Shen, Hongxia JinEMNLP 2020 · 4 citations
