ACL Findings2025
A Large and Balanced Corpus for Fine-grained Arabic Readability Assessment
Khalid N. Elmadani, Nizar Habash, Hanada Taha-Thomure
Abstract
This paper introduces the Balanced Arabic Readability Evaluation Corpus (BAREC), 1 a large-scale, fine-grained dataset for Arabic readability assessment. BAREC consists of 69,441 sentences spanning 1+ million words, carefully curated to cover 19 readability levels, from kindergarten to postgraduate comprehension. The corpus balances genre diversity, topical coverage, and target audiences, offering a comprehensive resource for evaluating Arabic text complexity. The corpus was fully manually annotated by a large team of annotators. The average pairwise inter-annotator agreement, measured by Quadratic Weighted Kappa, is 81.8%, reflecting a high level of substantial agreement. Beyond presenting the corpus, we benchmark automatic readability assessment across different granularity levels, comparing a range of techniques. Our results highlight the challenges and opportunities in Arabic readability modeling, demonstrating competitive performance across various methods. To support research and education, we make BAREC openly available, along with detailed annotation guidelines and benchmark results. 2
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on4
- Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic FeaturesBruce W. Lee, Yoo Sung Jang, Jason Hyung-Jong LeeEMNLP 2021 · 46 citations
- ArabicMMLU: Assessing Massive Multitask Language Understanding in ArabicFajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman et al.ACL Findings 2024 · 17 citations
- ARBERT & MARBERT: Deep Bidirectional Transformers for ArabicMuhammad Abdul-Mageed, AbdelRahim A. Elmadany, El Moatez Billah NagoudiACL 2021
- Making Better Mistakes: Leveraging Class Hierarchies With Deep NetworksLuca Bertinetto, Romain Müller, Konstantinos Tertikas, Sina Samangooei et al.CVPR 2020
Related papers
- ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability AssessmentTarek Naous, Michael J. Ryan, Anton Lavrouk, Mohit Chandra et al.EMNLP 2024 · 7 citations
- CEFR-Based Sentence Difficulty Annotation and AssessmentYuki Arase, Satoru Uchida, Tomoyuki KajiwaraEMNLP 2022 · 17 citations
- A Neural Pairwise Ranking Model for Readability AssessmentJustin Lee, Sowmya VajjalaACL Findings 2022 · 26 citations
- MedReadMe: A Systematic Study for Fine-grained Sentence Readability in Medical DomainChao Jiang, Wei XuEMNLP 2024 · 3 citations
- ORCA: A Challenging Benchmark for Arabic Language UnderstandingAbdelRahim A. Elmadany, El Moatez Billah Nagoudi, Muhammad Abdul-MageedACL Findings 2023 · 33 citations
