Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector

Chinese grammatical error correction (CGEC) faces serious overcorrection challenges when employing autoregressive generative models such as sequence-to-sequence (Seq2Seq) models and decoder-only large language models (LLMs). While previous methods aim to address overcorrection in Seq2Seq models, they are difficult to adapt to decoder-only LLMs. In this paper, we propose an alignment-enhanced corrector for the overcorrection problem that applies to both Seq2Seq models and decoderonly LLMs. Our method first trains a correction model to generate an initial correction of the source sentence. Then, we combine the source sentence with the initial correction and feed it through an alignment model for another round of correction, aiming to enforce the alignment model to focus on potential overcorrection. Moreover, to enhance the model's ability to identify nuances, we further explore the reverse alignment of the source sentence and the initial correction. Finally, we transfer the alignment knowledge from two alignment models to the correction model, instructing it on how to avoid overcorrection. Experimental results on three CGEC datasets demonstrate the effectiveness of our approach in alleviating overcorrection and improving overall performance. Existing CGEC methods can be mainly divided 042 into three categories: sequence-to-edit (Seq2Edit), 043 sequence-to-sequence (Seq2Seq), and decoder-044 only large language models (LLMs). Seq2Edit 045 methods treat CGEC as a sequence tagging task by 046 predicting token-level edit operations (Liang et al., 047 2020; Zhang et al., 2022a). While offering fast 048 inference and robust error detection, these meth-049 ods may compromise text fluency and exhibit weak 050 migration ability due to the reliance on language-051 specific vocabulary (Li et al., 2022). Seq2Seq meth-052 ods tackle CGEC using neural machine translation 053 techniques (Fu et al., 2018; Zhao and Wang, 2020) 054 and excel in generating fluent sentences but of-055 ten lack controllability. More recently, decoder-056 only LLMs have demonstrated breakthrough per-057 formance in various NLP tasks, showing significant 058 potential in CGEC (Fang et al., 2023; Qu and Wu, 059 2023). However, research suggests that decoder-060 only LLMs still fall short of surpassing lightweight 061 state-of-the-art models (Zhang et al., 2023).