Advancing Multi-Criteria Chinese Word Segmentation Through Criterion Classification and Denoising
Tzu-Hsuan Chou, Chun-Yi Lin, Hung-Yu Kao
Abstract
Recent research on multi-criteria Chinese word segmentation (MCCWS) mainly focuses on building complex private structures, adding more handcrafted features, or introducing complex optimization processes. In this work, we show that through a simple yet elegant inputhint-based MCCWS model, we can achieve state-of-the-art (SoTA) performances on several datasets simultaneously. We further propose a novel criterion-denoising objective that hurts slightly on F1 score but achieves SoTA recall on out-of-vocabulary words. Our result establishes a simple yet strong baseline for future MCCWS research. Source code is available at https://github.com/IKMLab/ MCCWS .
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 9d95c0ea-f4fd-421c-92a7-e1801eb0b58cBuilds on1
Related papers
- RethinkCWS: Is Chinese Word Segmentation a Solved Task?Jinlan Fu, Pengfei Liu, Qi Zhang, Xuanjing HuangEMNLP 2020 · 19 citations
- Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-SupervisionMieradilijiang Maimaiti, Yang Liu, Yuanhang Zheng, Gang Chen et al.EMNLP 2021 · 7 citations
- TopWORDS-Seg: Simultaneous Text Segmentation and Word Discovery for Open-Domain Chinese Texts via Bayesian InferenceChangzai Pan, Maosong Sun, Ke DengACL 2022 · 6 citations
- Attention Is All You Need for Chinese Word SegmentationSufeng Duan, Hai ZhaoEMNLP 2020 · 30 citations
- Improving Chinese Word Segmentation with Wordhood Memory NetworksYuanhe Tian, Yan Song, Fei Xia, Tong Zhang et al.ACL 2020 · 95 citations
