EMNLP Findings2023
Adaptive Textual Label Noise Learning based on Pre-trained Models
Shaohuan Cheng, Wenyu Chen, Mingsheng Fu, Xuanting Xie, Hong Qu
Abstract
The label noise in real-world scenarios is unpredictable and can even be a mixture of different types of noise. To meet this challenge, we develop an adaptive textual label noise learning framework based on pre-trained models, which consists of an adaptive warm-up stage followed by a hybrid training stage. Specifically, an early stopping method, relying solely on the training set, is designed to dynamically terminate the warm-up process based on the model's fit level to different noise scenarios. The hybrid training stage incorporates several generalization strategies to gradually correct mislabeled instances, thereby making better use of noisy data. Experiments on multiple datasets demonstrate that our approach performs on-par with or even better than the state-of-the-art methods in various noise scenarios, including scenarios with the mixture of multiple types of noise.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise cites DivideMix: Learning with Noisy Labels as Semi-supervised Learning
- Early-Learning Regularization Prevents Memorization of Noisy Labels cites Symmetric Cross Entropy for Robust Learning With Noisy Labels
- Early-Learning Regularization Prevents Memorization of Noisy Labels cites DivideMix: Learning with Noisy Labels as Semi-supervised Learning
- Memorisation versus Generalisation in Pre-trained Language Models cites Early-Learning Regularization Prevents Memorization of Noisy Labels
Builds on10
- DivideMix: Learning with Noisy Labels as Semi-supervised LearningJunnan Li, Richard Socher, Steven C. H. HoiICLR 2020 · 1,326 citations
- Symmetric Cross Entropy for Robust Learning With Noisy LabelsYisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo et al.ICCV 2019 · 1,125 citations
- Early-Learning Regularization Prevents Memorization of Noisy LabelsSheng Liu, Jonathan Niles-Weed, Narges Razavian, Carlos Fernandez-GrandaNeurIPS 2020 · 798 citations
- Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label NoisePengfei Chen, Junjie Ye, Guangyong Chen, Jingwei Zhao et al.AAAI 2021 · 156 citations
- Memorisation versus Generalisation in Pre-trained Language ModelsMichael Tänzer, Sebastian Ruder, Marek ReiACL 2022 · 59 citations
Related papers
- Learning with Feature-Dependent Label Noise: A Progressive ApproachYikai Zhang, Songzhu Zheng, Pengxiang Wu, Mayank Goswami et al.ICLR 2021 · 184 citations
- Learning under Temporal Label NoiseSujay Nagaraj, Walter Gerych, Sana Tonekaboni, Anna Goldenberg et al.ICLR 2025
- Noise-Robust Fine-Tuning of Pretrained Language Models via External GuidanceSong Wang, Zhen Tan, Ruocheng Guo, Jundong LiEMNLP Findings 2023 · 12 citations
- Enhancing Multi-Label Text Classification under Label-Dependent Noise: A Label-Specific Denoising FrameworkPengyu Xu, Liping Jing, Jian YuEMNLP Findings 2024 · 2 citations
- Early Stopping Against Label Noise Without Validation DataSuqin Yuan, Lei Feng, Tongliang LiuICLR 2024 · 39 citations
