EMNLP Findings2025
The Progress Illusion: Revisiting meta-evaluation standards of LLM evaluators
Tianruo Rose Xu, Vedant Gaur, Liu Leqi, Tanya Goyal
Abstract
LLM judges have gained popularity as an inexpensive and performant substitute for human evaluation. However, we observe that the meta-evaluation setting in which the reliability of these LLM evaluators is established is substantially different from their use in model development. To address this, we revisit metaevaluations of LLM evaluators under a setting that more closely aligns with practice by examining evaluators' ability to distinguish test system pairs that are closer in capability. Our fine-grained approach shows that all LLM evaluator's correlations with human judgments are concerningly low when the models perform similarly, showcasing a key limitation of current norms. Equipped with this better methodology, we next analyze the impact that the choice of the reference model makes to LLMas-a-judge evaluator performance. We show that single-reference evaluators only perform well at ranking test systems that fall within particular capability ranges, even if the standard meta-evaluation reports high overall correlation. Taken together, our analysis shows critical issues with current LLM meta-evaluation and recommend avenues for improvement.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference cites Measuring Massive Multitask Language Understanding
- Evaluating Large Language Models at Evaluating Instruction Following cites Measuring Massive Multitask Language Understanding
- Evaluating Large Language Models at Evaluating Instruction Following cites AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
- Evaluating Large Language Models at Evaluating Instruction Following cites Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models
- Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models cites AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
Builds on8
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human PreferenceWei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos et al.ICML 2024 · 1,212 citations
- AlpacaFarm: A Simulation Framework for Methods that Learn from Human FeedbackYann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang et al.NeurIPS 2023 · 948 citations
- Prometheus: Inducing Fine-Grained Evaluation Capability in Language ModelsSeungone Kim, Jamin Shin, Yejin Choi, Joel Jang et al.ICLR 2024 · 468 citations
- Evaluating Large Language Models at Evaluating Instruction FollowingZhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng et al.ICLR 2024 · 299 citations
Related papers
- Do Before You Judge: Self-Reference as a Pathway to Better LLM EvaluationWei-Hsiang Lin, Sheng-Lun Wei, Hen-Hsen Huang, Hsin-Hsi ChenEMNLP Findings 2025
- A Dual-Perspective NLG Meta-Evaluation Framework with Automatic Benchmark and Better InterpretabilityXinyu Hu, Mingqi Gao, Li Lin, Zhenghan Yu et al.ACL 2025
- An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4Hui Huang, Xingyuan Bu, Hongli Zhou, Yingqi Qu et al.ACL Findings 2025 · 43 citations
- JuStRank: Benchmarking LLM Judges for System RankingAriel Gera, Odellia Boni, Yotam Perlitz, Roy Bar-Haim et al.ACL 2025
- WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development QualityChunyang Li, Yilun Zheng, Xinting Huang, Tianqing Fang et al.ICLR 2026 · 14 citations
