To Err Is Human, but Llamas Can Learn It Too

This study explores enhancing grammatical error correction (GEC) through artificial error generation (AEG) using large language models (LLMs). Specifically, we fine-tune Llama 2-based LLMs for error generation and find that this approach yields synthetic errors akin to human errors. Next, we train GEC Llama models with the help of these artificial errors and outperform previous state-of-the-art error correction models, with gains ranging between 0.8 and 6 F 0.5 points across all tested languages (German, Ukrainian, and Estonian). Moreover, we demonstrate that generating errors by finetuning smaller sequence-to-sequence models and prompting large commercial LLMs (GPT-3.5 and GPT-4) also results in synthetic errors beneficially affecting error generation models. We openly release trained models for error generation and correction and all the synthesized error datasets for the covered languages.