EMNLP Findings2020
Pretrained Language Models for Dialogue Generation with Multiple Input Sources
Yu Cao, Wei Bi, Meng Fang, Dacheng Tao
Abstract
Large-scale pretrained language models have achieved outstanding performance on natural language understanding tasks. However, it is still under investigating how to apply them to dialogue generation tasks, especially those with responses conditioned on multiple sources. Previous work simply concatenates all input sources or averages information from different input sources. In this work, we study dialogue models with multiple input sources adapted from the pretrained language model GPT2. We explore various methods to fuse multiple separate attention information corresponding to different sources. Our experimental results show that proper fusion methods deliver higher relevance with dialogue history than simple fusion baselines.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
4 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation cites A Model-agnostic Data Manipulation Method for Persona-based Dialogue Generation
- SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation cites A Pre-Training Based Personalized Dialogue Generation Model with Persona-Sparse Data
Cited by top-tier papers4
- DialogLM: Pre-trained Model for Long Dialogue Understanding and SummarizationMing Zhong, Yang Liu, Yichong Xu, Chenguang Zhu et al.AAAI 2022 · 150 citations
- SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluationJunkai Zhou, Liang Pang, Huawei Shen, Xueqi ChengACL 2023 · 6 citations
- TGEA: An Error-Annotated Dataset and Benchmark Tasks for TextGeneration from Pretrained Language ModelsJie He, Bo Peng, Yi Liao, Qun Liu et al.ACL 2021
- A Model-agnostic Data Manipulation Method for Persona-based Dialogue GenerationYu Cao, Wei Bi, Meng Fang, Shuming Shi et al.ACL 2022
Builds on1
Related papers
- Multi-Stage Prompting for Knowledgeable Dialogue GenerationZihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye et al.ACL Findings 2022 · 45 citations
- Knowledge-Grounded Dialogue Generation with Pre-trained Language ModelsXueliang Zhao, Wei Wu, Can Xu, Chongyang Tao et al.EMNLP 2020 · 153 citations
- Collaborative Reasoning on Multi-Modal Semantic Graphs for Video-Grounded Dialogue GenerationXueliang Zhao, Yuxuan Wang, Chongyang Tao, Chenshuo Wang et al.EMNLP Findings 2022 · 5 citations
- Multi-Source Probing for Open-Domain Conversational UnderstandingYuanxi Li, Hao Zhou, Jie Zhou, Minlie HuangEMNLP 2023
- Generating Images with Multimodal Language ModelsJing Yu Koh, Daniel Fried, Russ SalakhutdinovNeurIPS 2023 · 403 citations
