Lune

ACL Findings2023

Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding

Mirac Suzgun, Luke Melas-Kyriazi, Dan Jurafsky

Abstract

In open-ended natural-language generation, existing text decoding methods typically struggle to produce text which is both diverse and highquality. Greedy and beam search are known to suffer from text degeneration and linguistic diversity issues, while temperature, top-k, and nucleus sampling yield diverse but often lowerquality outputs. In this work, we build upon Minimum Bayes Risk Decoding (MBRD), a family of decoding methods based on Bayesian risk minimization, to address this diversityquality trade-off. Inspired by the principle of the wisdom of the crowd, MBRD seeks to select a candidate from a pool of candidates that has the least expected risk under a generative model according to a given utility function. The crowd of candidates serves as an approximation for the distribution over human-generated references. We show that MBRD generalizes numerous decoding methods, including majority voting, and can be used as a drop-in replacement for existing sampling methods. Across a wide range of tasks-such as summarization, datato-text, translation, and textual style transfer-MBRD yields 3-7 ROUGE and BLEU point improvements, including state-of-the-art results on WebNLG and WMT'16. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

14 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Improving Minimum Bayes Risk Decoding with Multi-Prompt cites BERTScore: Evaluating Text Generation with BERT
  • Improving Minimum Bayes Risk Decoding with Multi-Prompt cites Finetuned Language Models are Zero-Shot Learners
  • Guiding Large Language Models via Directional Stimulus Prompting cites Language Models are Few-Shot Learners
  • Guiding Large Language Models via Directional Stimulus Prompting cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability cites BERTScore: Evaluating Text Generation with BERT
  • Language Models are Few-Shot Learners cites The Curious Case of Neural Text Degeneration
  • Model-Based Minimum Bayes Risk Decoding for Text Generation cites The Curious Case of Neural Text Degeneration
  • Finetuned Language Models are Zero-Shot Learners cites Language Models are Few-Shot Learners
  • Finetuned Language Models are Zero-Shot Learners cites Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers14

  1. Guiding Large Language Models via Directional Stimulus PromptingZekun Li, Baolin Peng, Pengcheng He, Michel Galley et al.NeurIPS 2023 · 163 citations
  2. Ensemble Distillation for Unsupervised Constituency ParsingBehzad Shayegh, Yanshuai Cao, Xiaodan Zhu, Jackie C. K. Cheung et al.ICLR 2024 · 10 citations
  3. Model-Based Minimum Bayes Risk Decoding for Text GenerationYuu Jinnai, Tetsuro Morimura, Ukyo Honda, Kaito Ariu et al.ICML 2024 · 9 citations
  4. What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production VariabilityMario Giulianelli, Joris Baan, Wilker Aziz, Raquel Fernández et al.EMNLP 2023 · 5 citations
  5. Improving Minimum Bayes Risk Decoding with Multi-PromptDavid Heineman, Yao Dou, Wei XuEMNLP 2024 · 1 citation

Builds on15

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Chain-of-Thought Prompting Elicits Reasoning in Large Language ModelsJason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma et al.NeurIPS 2022 · 22,562 citations
  3. BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
  4. Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
  5. The Curious Case of Neural Text DegenerationAri Holtzman, Jan Buys, Li Du, Maxwell Forbes et al.ICLR 2020 · 4,112 citations

Related papers

  1. Generating Diverse and High-Quality Texts by Minimum Bayes Risk DecodingYuu Jinnai, Ukyo Honda, Tetsuro Morimura, Peinan ZhangACL Findings 2024 · 2 citations
  2. Uncertainty-Aware Decoding with Minimum Bayes RiskNico Daheim, Clara Meister, Thomas Möllenhoff, Iryna GurevychICLR 2025
  3. Case-Based Decision-Theoretic Decoding with Quality MemoriesHiroyuki Deguchi, Masaaki NagataEMNLP 2025
  4. Diversity Explains Inference Scaling Laws: Through a Case Study of Minimum Bayes Risk DecodingHidetaka Kamigaito, Hiroyuki Deguchi, Yusuke Sakai, Katsuhiko Hayashi et al.ACL 2025
  5. Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine TranslationMathias Müller, Rico SennrichACL 2021
Dusk over the sea between two cliffs drawn in fine vertical lines