Lune

ACL Findings2023

Joint Generator-Ranker Learning for Natural Language Generation

Weizhou Shen, Yeyun Gong, Yelong Shen, Song Wang, Xiaojun Quan, Nan Duan, Weizhu Chen

Abstract

Generate-then-rank is a widely used mechanism for text generation, where a generator produces multiple text candidates and a ranker chooses the best one among the text candidates. However, existing methods usually train the generator and the ranker individually, neglecting the mutual feedback that could further enhance the generation quality. To tackle this limitation, we propose JGR, a novel joint training algorithm that integrates the generator and the ranker in a single framework. JGR optimizes the generator with a hybrid objective that combines data likelihood and ranker reward, and trains the ranker with a contrastive loss that compares the generator outputs. By iteratively updating the generator and the ranker, JGR can effectively harmonize their learning and enhance their quality jointly. We evaluate JGR on various text generation tasks and demonstrate that it surpasses existing methods on four public datasets across three common generation scenarios. Our code and models are publicly available at https://github.com/ microsoft/ProphetNet/tree/master/JGR.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Lift Yourself Up: Retrieval-augmented Text Generation with Self-Memory cites BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
  • Lift Yourself Up: Retrieval-augmented Text Generation with Self-Memory cites PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. Lift Yourself Up: Retrieval-augmented Text Generation with Self-MemoryXin Cheng, Di Luo, Xiuying Chen, Lemao Liu et al.NeurIPS 2023 · 177 citations
  2. Metasql: A Generate-Then-Rank Framework for Natural Language to SQL TranslationYuankai Fan, Zhenying He, Tonghui Ren, Can Huang et al.ICDE 2024 · 23 citations

Builds on18

  1. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  2. PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive SummarizationJingqing Zhang, Yao Zhao, Mohammad Saleh, Peter J. LiuICML 2020 · 2,453 citations
  3. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
  4. CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement LearningHung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese et al.NeurIPS 2022 · 571 citations
  5. Extractive Summarization as Text MatchingMing Zhong, Pengfei Liu, Yiran Chen, Danqing Wang et al.ACL 2020 · 410 citations

Related papers

  1. Adversarial Text Generation via Sequence Contrast DiscriminationKe Wang, Xiaojun WanEMNLP Findings 2020 · 3 citations
  2. Learning to Rank in Generative RetrievalYongqi Li, Nan Yang, Liang Wang, Furu Wei et al.AAAI 2024 · 83 citations
  3. Distillation Enhanced Generative RetrievalYongqi Li, Zhen Zhang, Wenjie Wang, Liqiang Nie et al.ACL Findings 2024 · 6 citations
  4. URG: A Unified Ranking and Generation Method for Ensembling Language ModelsBo Lv, Chen Tang, Yanan Zhang, Xin Liu et al.ACL Findings 2024 · 1 citation
  5. Self-Adversarial Learning with Comparative Discrimination for Text GenerationWangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei et al.ICLR 2020 · 20 citations
Dusk over the sea between two cliffs drawn in fine vertical lines