Lune

EMNLP Findings2021

Towards Realistic Single-Task Continuous Learning Research for NER

Justin Payan, Yuval Merhav, He Xie, Satyapriya Krishna, Anil Ramakrishna, Mukund Sridhar, Rahul Gupta

Abstract

There is an increasing interest in continuous learning (CL), as data privacy is becoming a priority for real-world machine learning applications. Meanwhile, there is still a lack of academic NLP benchmarks that are applicable for realistic CL settings, which is a major challenge for the advancement of the field. In this paper we discuss some of the unrealistic data characteristics of public datasets, study the challenges of realistic single-task continuous learning as well as the effectiveness of data rehearsal as a way to mitigate accuracy loss. We construct a CL NER dataset from an existing publicly available dataset and release it along with the code to the research community 1 .

Work completed while first author was an intern at Amazon Alexa AI.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Unify Named Entity Recognition Scenarios via Contrastive Real-Time Updating Prototype cites Code and Named Entity Recognition in StackOverflow
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Unify Named Entity Recognition Scenarios via Contrastive Real-Time Updating PrototypeYanhe Liu, Peng Wang, Wenjun Ke, Guozheng Li et al.AAAI 2024 · 7 citations

Builds on2

  1. LAMOL: LAnguage MOdeling for Lifelong Language LearningFan-Keng Sun, Cheng-Hao Ho, Hung-Yi LeeICLR 2020 · 247 citations
  2. Code and Named Entity Recognition in StackOverflowJeniya Tabassum, Mounica Maddela, Wei Xu, Alan RitterACL 2020 · 9 citations

Related papers

  1. TiC-CLIP: Continual Training of CLIP ModelsSaurabh Garg, Mehrdad Farajtabar, Hadi Pouransari, Raviteja Vemulapalli et al.ICLR 2024 · 44 citations
  2. CITB: A Benchmark for Continual Instruction TuningZihan Zhang, Meng Fang, Ling Chen, Mohammad-Reza Namazi-RadEMNLP Findings 2023 · 8 citations
  3. Rehearsal revealed: The limits and merits of revisiting samples in continual learningEli Verwimp, Matthias De Lange, Tinne TuytelaarsICCV 2021 · 121 citations
  4. Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized RehearsalJianheng Huang, Leyang Cui, Ante Wang, Chengyi Yang et al.ACL 2024 · 13 citations
  5. Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language ModelsZangwei Zheng, Mingyuan Ma, Kai Wang, Ziheng Qin et al.ICCV 2023 · 133 citations
Dusk over the sea between two cliffs drawn in fine vertical lines