ACL Findings2024
Rich Semantic Knowledge Enhanced Large Language Models for Few-shot Chinese Spell Checking
Abstract
Chinese Spell Checking (CSC) is a widely used technology, which plays a vital role in speech to text (STT) and optical character recognition (OCR). Most of the existing CSC approaches relying on BERT architecture achieve excellent performance. However, limited by the scale of the foundation model, BERT-based method does not work well in few-shot scenarios, showing certain limitations in practical applications. In this paper, we explore using an incontext learning method named RS-LLM (Rich Semantic based LLMs) to introduce large language models (LLMs) as the foundation model. Besides, we study the impact of introducing various Chinese rich semantic information in our framework. We found that by introducing a small number of specific Chinese rich semantic structures, LLMs achieve better performance than most of the BERT-based model on few-shot CSC task. Furthermore, we conduct experiments on multiple datasets, and the experimental results verified the superiority of our proposed framework.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking cites Spelling Error Correction with Soft-Masked BERT
- Mixture of Small and Large Models for Chinese Spelling Check cites Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking
- Mixture of Small and Large Models for Chinese Spelling Check cites A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Models
- Mixture of Small and Large Models for Chinese Spelling Check cites Spelling Error Correction with Soft-Masked BERT
- Mixture of Small and Large Models for Chinese Spelling Check cites ARM: An Alignment-and-Replacement Module for Chinese Spelling Check Based on LLMs
- A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Models cites Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking
- A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Models cites Spelling Error Correction with Soft-Masked BERT
- An Explanation of In-context Learning as Implicit Bayesian Inference cites Language Models are Few-Shot Learners
- Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers cites An Explanation of In-context Learning as Implicit Bayesian Inference
- ARM: An Alignment-and-Replacement Module for Chinese Spelling Check Based on LLMs cites Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking
- ARM: An Alignment-and-Replacement Module for Chinese Spelling Check Based on LLMs cites Spelling Error Correction with Soft-Masked BERT
Cited by top-tier papers3
- Mixture of Small and Large Models for Chinese Spelling CheckZiheng Qiao, Houquan Zhou, Zhenghua LiACL 2025 · 4 citations
- ARM: An Alignment-and-Replacement Module for Chinese Spelling Check Based on LLMsChangchun Liu, Kai Zhang, Junzhe Jiang, Zirui Liu et al.EMNLP 2024 · 3 citations
- A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language ModelsHouquan Zhou, Zhenghua Li, Bo Zhang, Chen Li et al.EMNLP 2024 · 2 citations
Builds on12
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- An Explanation of In-context Learning as Implicit Bayesian InferenceSang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu MaICLR 2022 · 1,030 citations
- Spelling Error Correction with Soft-Masked BERTShaohua Zhang, Haoran Huang, Jicong Liu, Hang LiACL 2020 · 204 citations
- Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-OptimizersDamai Dai, Yutao Sun, Li Dong, Yaru Hao et al.ACL Findings 2023 · 154 citations
- Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell CheckingHeng-Da Xu, Zhongli Li, Qingyu Zhou, Chao Li et al.ACL Findings 2021 · 91 citations
Related papers
- C-LLM: Learn to Check Chinese Spelling Errors Character by CharacterKunting Li, Yong Hu, Liang He, Fandong Meng et al.EMNLP 2024 · 9 citations
- MDCSpell: A Multi-task Detector-Corrector Framework for Chinese Spelling CorrectionChenxi Zhu, Ziqiang Ying, Boyu Zhang, Feng MaoACL Findings 2022 · 39 citations
- Chinese Spelling Correction as Rephrasing Language ModelLinfeng Liu, Hongqiu Wu, Hai ZhaoAAAI 2024 · 36 citations
- Rethinking Masked Language Modeling for Chinese Spelling CorrectionHongqiu Wu, Shaohua Zhang, Yuchen Zhang, Hai ZhaoACL 2023 · 21 citations
- UMRSpell: Unifying the Detection and Correction Parts of Pre-trained Models towards Chinese Missing, Redundant, and Spelling CorrectionZheyu He, Yujin Zhu, Linlin Wang, Liang XuACL 2023 · 8 citations
