Lemmatization as a Classification Task: Results from Arabic across Multiple Genres
Mostafa Saeed, Nizar Habash
Abstract
Lemmatization is crucial for NLP tasks in morphologically rich languages with ambiguous orthography like Arabic, but existing tools face challenges due to inconsistent standards and limited genre coverage. This paper introduces two novel approaches that frame lemmatization as classification into a Lemma-POS-Gloss (LPG) tagset, leveraging machine translation and semantic clustering. We also present a new Arabic lemmatization test set covering diverse genres, standardized alongside existing datasets. We evaluate character-level sequenceto-sequence models, which perform competitively and offer complementary value, but are limited to lemma prediction (not LPG) and prone to hallucinating implausible forms. Our results show that classification and clustering yield more robust, interpretable outputs, setting new benchmarks for Arabic lemmatization. B License In Table 12 , we list the license of the data and tools used in this work. All of them are used under their intended use.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Related papers
- Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological TaggingNasser Zalmout, Nizar HabashACL 2020 · 3 citations
- Advancements in Arabic Grammatical Error Detection and Correction: An Empirical InvestigationBashar Alhafni, Go Inoue, Christian Khairallah, Nizar HabashEMNLP 2023 · 10 citations
- AraT5: Text-to-Text Transformers for Arabic Language GenerationEl Moatez Billah Nagoudi, AbdelRahim A. Elmadany, Muhammad Abdul-MageedACL 2022 · 175 citations
- TAMS: Translation-Assisted Morphological SegmentationEnora Rice, Ali Marashian, Luke Gessler, Alexis Palmer et al.ACL 2024
- Transforming Sequence Tagging Into A Seq2Seq TaskKarthik Raman, Iftekhar Naim, Jiecao Chen, Kazuma Hashimoto et al.EMNLP 2022 · 12 citations
