EMNLP Findings2020
Adversarial Text Generation via Sequence Contrast Discrimination
Abstract
In this paper, we propose a sequence contrast loss driven text generation framework, which learns the difference between real texts and generated texts and uses that difference. Specifically, our discriminator contains a discriminative sequence generator instead of a binary classifier, and measures the 'relative realism' of generated texts against real texts by making use of them simultaneously. Moreover, our generator uses discriminative sequences to directly improve itself, which not only replaces the gradient propagation process from the discriminator to the generator, but also avoids the time-consuming sampling process of estimating rewards in some previous methods. We conduct extensive experiments with various metrics, substantiating that our framework brings improvements in terms of training stability and the quality of generated texts.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Data-Efficient Image Recognition with Contrastive Predictive Coding cites Momentum Contrast for Unsupervised Visual Representation Learning
- Momentum Contrast for Unsupervised Visual Representation Learning cites Data-Efficient Image Recognition with Contrastive Predictive Coding
- A Simple Framework for Contrastive Learning of Visual Representations cites Data-Efficient Image Recognition with Contrastive Predictive Coding
- A Simple Framework for Contrastive Learning of Visual Representations cites Momentum Contrast for Unsupervised Visual Representation Learning
- Momentum Contrast for Unsupervised Visual Representation Learning cites A Simple Framework for Contrastive Learning of Visual Representations
Cited by top-tier papers1
Builds on3
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- Data-Efficient Image Recognition with Contrastive Predictive CodingOlivier J. HénaffICML 2020 · 1,553 citations
- Momentum Contrast for Unsupervised Visual Representation LearningKaiming He, Haoqi Fan, Yuxin Wu, Saining Xie et al.CVPR 2020
Related papers
- Self-Adversarial Learning with Comparative Discrimination for Text GenerationWangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei et al.ICLR 2020 · 20 citations
- CoNT: Contrastive Neural Text GenerationChenxin An, Jiangtao Feng, Kai Lv, Lingpeng Kong et al.NeurIPS 2022 · 37 citations
- Counter-Contrastive Learning for Language GANsYekun Chai, Haidong Zhang, Qiyue Yin, Junge ZhangEMNLP Findings 2021 · 4 citations
- TextGAIL: Generative Adversarial Imitation Learning for Text GenerationQingyang Wu, Lei Li, Zhou YuAAAI 2021 · 54 citations
- Reading and Writing: Discriminative and Generative Modeling for Self-Supervised Text RecognitionMingkun Yang, Minghui Liao, Pu Lu, Jing Wang et al.ACM MM 2022 · 69 citations
