The Authenticity Gap in Human Evaluation
Abstract
Human ratings are the gold standard in NLG evaluation. The standard protocol is to collect ratings of generated text, average across annotators, and rank NLG systems by their average scores. However, little consideration has been given as to whether this approach faithfully captures human preferences. Analyzing this standard protocol through the lens of utility theory in economics, we identify the implicit assumptions it makes about annotators. These assumptions are often violated in practice, in which case annotator ratings cease to reflect their preferences. The most egregious violations come from using Likert scales, which provably reverse the direction of the true preference in certain cases. We suggest improvements to the standard protocol to make it more theoretically sound, but even in its improved form, it cannot be used to evaluate open-ended tasks like story generation. For the latter, we propose a new human evaluation protocol called system-level probabilistic assessment (SPA). When human evaluation of stories is done with SPA, we can recover the ordering of GPT-3 models by size, with statistically significant results. However, when human evaluation is done with the standard protocol, less than half of the expected preferences can be recovered (e.g., there is no significant difference between curie and davinci, despite using a highly powered test).
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
6 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Large-Scale Correlation Analysis of Automated Metrics for Topic Models cites With Little Power Comes Great Responsibility
- Large-Scale Correlation Analysis of Automated Metrics for Topic Models cites All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text
- Language Model Alignment in Multilingual Trolley Problems cites Language Models are Few-Shot Learners
- Language Models are Few-Shot Learners cites Automatic Detection of Generated Text is Easiest when Humans are Fooled
- Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages cites All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text
- All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text cites Automatic Detection of Generated Text is Easiest when Humans are Fooled
- All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text cites With Little Power Comes Great Responsibility
Cited by top-tier papers6
- Large-Scale Correlation Analysis of Automated Metrics for Topic ModelsJia Peng Lim, Hady W. LauwACL 2023 · 13 citations
- Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different LanguagesShreyan Biswas, Alexander Erlei, Ujwal GadirajuCHI 2025 · 11 citations
- Preconditioned Test-Time Adaptation for Out-of-Distribution Debiasing in Narrative GenerationHanwen Shen, Ting Ying, Jiajie Lu, Shanshan WangACL 2026 · 5 citations
- A SMART Mnemonic Sounds like "Glue Tonic": Mixing LLMs with Student Feedback to Make Mnemonic Learning StickNishant Balepur, Matthew Shu, Alexander Miserlis Hoyle, Alison Robey et al.EMNLP 2024 · 2 citations
- Language Model Alignment in Multilingual Trolley ProblemsZhijing Jin, Max Kleiman-Weiner, Giorgio Piatti, Sydney Levine et al.ICLR 2025 · 1 citation
Builds on4
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- With Little Power Comes Great ResponsibilityDallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia et al.EMNLP 2020 · 76 citations
- Automatic Detection of Generated Text is Easiest when Humans are FooledDaphne Ippolito, Daniel Duckworth, Chris Callison-Burch, Douglas EckACL 2020 · 21 citations
- All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated TextElizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong et al.ACL 2021
Related papers
- Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation ModelsWangchunshu Zhou, Ke XuAAAI 2020 · 49 citations
- Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language ModelsHritik Bansal, John Dang, Aditya GroverICLR 2024 · 28 citations
- Can Large Language Models Be an Alternative to Human Evaluations?David Cheng-Han Chiang, Hung-yi LeeACL 2023 · 254 citations
- G-Eval: NLG Evaluation using Gpt-4 with Better Human AlignmentYang Liu, Dan Iter, Yichong Xu, Shuohang Wang et al.EMNLP 2023 · 549 citations
- Intrinsic Task-based Evaluation for Referring Expression GenerationGuanyi Chen, Fahime Same, Kees van DeemterACL 2024
