Lune

EMNLP Findings2020

Pretrained Language Models for Dialogue Generation with Multiple Input Sources

Yu Cao, Wei Bi, Meng Fang, Dacheng Tao

Abstract

Large-scale pretrained language models have achieved outstanding performance on natural language understanding tasks. However, it is still under investigating how to apply them to dialogue generation tasks, especially those with responses conditioned on multiple sources. Previous work simply concatenates all input sources or averages information from different input sources. In this work, we study dialogue models with multiple input sources adapted from the pretrained language model GPT2. We explore various methods to fuse multiple separate attention information corresponding to different sources. Our experimental results show that proper fusion methods deliver higher relevance with dialogue history than simple fusion baselines.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation cites A Model-agnostic Data Manipulation Method for Persona-based Dialogue Generation
  • SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation cites A Pre-Training Based Personalized Dialogue Generation Model with Persona-Sparse Data
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. DialogLM: Pre-trained Model for Long Dialogue Understanding and SummarizationMing Zhong, Yang Liu, Yichong Xu, Chenguang Zhu et al.AAAI 2022 · 150 citations
  2. SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluationJunkai Zhou, Liang Pang, Huawei Shen, Xueqi ChengACL 2023 · 6 citations
  3. TGEA: An Error-Annotated Dataset and Benchmark Tasks for TextGeneration from Pretrained Language ModelsJie He, Bo Peng, Yi Liao, Qun Liu et al.ACL 2021
  4. A Model-agnostic Data Manipulation Method for Persona-based Dialogue GenerationYu Cao, Wei Bi, Meng Fang, Shuming Shi et al.ACL 2022

Builds on1

  1. A Pre-Training Based Personalized Dialogue Generation Model with Persona-Sparse DataYinhe Zheng, Rongsheng Zhang, Minlie Huang, Xiaoxi MaoAAAI 2020 · 173 citations

Related papers

  1. Multi-Stage Prompting for Knowledgeable Dialogue GenerationZihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye et al.ACL Findings 2022 · 45 citations
  2. Knowledge-Grounded Dialogue Generation with Pre-trained Language ModelsXueliang Zhao, Wei Wu, Can Xu, Chongyang Tao et al.EMNLP 2020 · 153 citations
  3. Collaborative Reasoning on Multi-Modal Semantic Graphs for Video-Grounded Dialogue GenerationXueliang Zhao, Yuxuan Wang, Chongyang Tao, Chenshuo Wang et al.EMNLP Findings 2022 · 5 citations
  4. Multi-Source Probing for Open-Domain Conversational UnderstandingYuanxi Li, Hao Zhou, Jie Zhou, Minlie HuangEMNLP 2023
  5. Generating Images with Multimodal Language ModelsJing Yu Koh, Daniel Fried, Russ SalakhutdinovNeurIPS 2023 · 403 citations
Dusk over the sea between two cliffs drawn in fine vertical lines