OCR Post Correction for Endangered Language Texts
Shruti Rijhwani, Antonios Anastasopoulos, Graham Neubig
Abstract
There is little to no data available to build natural language processing models for most endangered languages. However, textual data in these languages often exists in formats that are not machine-readable, such as paper books and scanned images. In this work, we address the task of extracting text from these resources. We create a benchmark dataset of transcriptions for scanned books in three critically endangered languages and present a systematic analysis of how general-purpose OCR tools are not robust to the data-scarce setting of endangered languages. We develop an OCR postcorrection method tailored to ease training in this data-scarce setting, reducing the recognition error rate by 34% on average across the three languages. 1
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext b79faefb-4093-4eb2-8048-6dddadc8d3c7Cited by top-tier papers10
- Expanding Pretrained Models to Thousands More Languages via Lexicon-based AdaptationXinyi Wang, Sebastian Ruder, Graham NeubigACL 2022 · 73 citations
- How can NLP Help Revitalize Endangered Languages? A Case Study and Roadmap for the Cherokee LanguageShiyue Zhang, Benjamin Frey, Mohit BansalACL 2022 · 46 citations
- Robust Open-Vocabulary Translation from Visual Text RepresentationsElizabeth Salesky, David Etter, Matt PostEMNLP 2021 · 33 citations
- Post-OCR Document Correction with Large Ensembles of Character Sequence-to-Sequence ModelsJuan Antonio Ramirez-Orta, Eduardo Xamena, Ana Gabriela Maguitman, Evangelos E. Milios et al.AAAI 2022 · 20 citations
- Extracting Training Data From Document-Based VQA ModelsFrancesco Pinto, Nathalie Rauschmayr, Florian Tramèr, Philip Torr et al.ICML 2024 · 7 citations
Builds on1
Related papers
- NusaAksara: A Multimodal and Multilingual Benchmark for Preserving Indonesian Indigenous ScriptsMuhammad Farid Adilazuarda, Musa Izzanardi Wijanarko, Lucky Susanto, Khumaisa Nur'aini et al.ACL 2025
- A Benchmark for Learning to Translate a New Language from One Grammar BookGarrett Tanzer, Mirac Suzgun, Eline Visser, Dan Jurafsky et al.ICLR 2024 · 97 citations
- LogogramNLP: Comparing Visual and Textual Representations of Ancient Logographic Writing Systems for NLPDanlu Chen, Freda Shi, Aditi Agarwal, Jacobo Myerston et al.ACL 2024
- Optimized Tokenization for Transcribed Error CorrectionTomer Wullach, Shlomo E. ChazanEMNLP 2023 · 1 citation
- TAMS: Translation-Assisted Morphological SegmentationEnora Rice, Ali Marashian, Luke Gessler, Alexis Palmer et al.ACL 2024
