Global Attention Decoder for Chinese Spelling Error Correction

Recent progress has been made in using BERT framework for Chinese spelling error correction (CSC). However, most existing methods correct words based on local contextual information, without considering the influence of error words in sentences. Imposing attention on error contextual information could mislead and decrease the overall performance of CSC. To address this issue, we propose a Global Attention Decoder (GAD) approach for CSC. Specifically, the proposed method learns the global relationship of the potential correct input characters and the candidates of potential error characters. Rich global contextual information is obtained to alleviate the impact of the local error contextual information. In addition, a BERT with Confusion set guided Replacement Strategy (BERT CRS) is designed to narrow the gap between BERT and CSC. The candidates generated by BERT CRS covering the correct character with more than 99.9% probability. To demonstrate the effectiveness of our proposed framework, we test our method on three human-annotated datasets. The experimental results show that our approach outperforms all competitor models by a large margin of up to 6.2%, achieving state-of-the-art methods on all datasets.