Lune

ACL Findings2025

A Large and Balanced Corpus for Fine-grained Arabic Readability Assessment

Khalid N. Elmadani, Nizar Habash, Hanada Taha-Thomure

Abstract

This paper introduces the Balanced Arabic Readability Evaluation Corpus (BAREC), 1 a large-scale, fine-grained dataset for Arabic readability assessment. BAREC consists of 69,441 sentences spanning 1+ million words, carefully curated to cover 19 readability levels, from kindergarten to postgraduate comprehension. The corpus balances genre diversity, topical coverage, and target audiences, offering a comprehensive resource for evaluating Arabic text complexity. The corpus was fully manually annotated by a large team of annotators. The average pairwise inter-annotator agreement, measured by Quadratic Weighted Kappa, is 81.8%, reflecting a high level of substantial agreement. Beyond presenting the corpus, we benchmark automatic readability assessment across different granularity levels, comparing a range of techniques. Our results highlight the challenges and opportunities in Arabic readability modeling, demonstrating competitive performance across various methods. To support research and education, we make BAREC openly available, along with detailed annotation guidelines and benchmark results. 2

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic cites ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on4

  1. Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic FeaturesBruce W. Lee, Yoo Sung Jang, Jason Hyung-Jong LeeEMNLP 2021 · 46 citations
  2. ArabicMMLU: Assessing Massive Multitask Language Understanding in ArabicFajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman et al.ACL Findings 2024 · 17 citations
  3. ARBERT & MARBERT: Deep Bidirectional Transformers for ArabicMuhammad Abdul-Mageed, AbdelRahim A. Elmadany, El Moatez Billah NagoudiACL 2021
  4. Making Better Mistakes: Leveraging Class Hierarchies With Deep NetworksLuca Bertinetto, Romain Müller, Konstantinos Tertikas, Sina Samangooei et al.CVPR 2020

Related papers

  1. ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability AssessmentTarek Naous, Michael J. Ryan, Anton Lavrouk, Mohit Chandra et al.EMNLP 2024 · 7 citations
  2. CEFR-Based Sentence Difficulty Annotation and AssessmentYuki Arase, Satoru Uchida, Tomoyuki KajiwaraEMNLP 2022 · 17 citations
  3. A Neural Pairwise Ranking Model for Readability AssessmentJustin Lee, Sowmya VajjalaACL Findings 2022 · 26 citations
  4. MedReadMe: A Systematic Study for Fine-grained Sentence Readability in Medical DomainChao Jiang, Wei XuEMNLP 2024 · 3 citations
  5. ORCA: A Challenging Benchmark for Arabic Language UnderstandingAbdelRahim A. Elmadany, El Moatez Billah Nagoudi, Muhammad Abdul-MageedACL Findings 2023 · 33 citations
Dusk over the sea between two cliffs drawn in fine vertical lines