EMNLP Findings2023
BLM-s/lE: A structured dataset of English spray-load verb alternations for testing generalization in LLMs
Giuseppe Samo, Vivi Nastase, Chunyang Jiang, Paola Merlo
摘要
Current NLP models appear to be achieving performance comparable to human capabilities on well-established benchmarks. New benchmarks are now necessary to test deeper layers of understanding of natural languages by these models.
问 Lune
尽管问，回答会引用原文。
Lune 会读完全文和相关的 Paper，每个回答都附上所依据的原文段落。
它引用的 Paper2
- ELECTRA: Pre-training Text Encoders as Discriminators Rather Than GeneratorsKevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. ManningICLR 2020 · 541 次引用
- Blackbird language matrices (BLM), a new task for rule-like generalization in neural networks: Can Large Language Models pass the test?Paola MerloEMNLP Findings 2023 · 2 次引用
相关 Paper
- Adversarial NLI: A New Benchmark for Natural Language UnderstandingYixin Nie, Adina Williams, Emily Dinan, Mohit Bansal 等ACL 2020 · 602 次引用
- AwarenessBench: Assessing Cognitive Capabilities of Language ModelsXiaojian Li, Rongwu Xu, Tianyun Zhang, Yue Wang 等ACL 2026
- In Benchmarks We Trust ... Or Not?Ine Gevers, Victor De Marez, Jens Van Nooten, Jens Lemmens 等EMNLP 2025 · 1 次引用
- What's the Best Place for an AI Conference, Vancouver or _______: Why Completing Comparative Questions is DifficultAvishai Zagoury, Einat Minkov, Idan Szpektor, William W. CohenAAAI 2021 · 6 次引用
- Do LLMs Really Struggle at NL-FOL Translation? Revealing Their Strengths via a Novel Benchmarking StrategyAndrea Brunello, Luca Geatti, Michele Mignani, Angelo Montanari 等AAAI 2026
