Lune

ACL Findings2024

Incremental Sequence Labeling: A Tale of Two Shifts

Shengjie Qiu, Junhao Zheng, Zhen Liu, Yicheng Luo, Qianli Ma

Abstract

The incremental sequence labeling task involves continuously learning new classes over time while retaining knowledge of the previous ones. Our investigation identifies two significant semantic shifts: E2O (where the model mislabels an old entity as a non-entity) and O2E (where the model labels a non-entity or old entity as a new entity). Previous research has predominantly focused on addressing the E2O problem, neglecting the O2E issue. This negligence results in a model bias towards classifying new data samples as belonging to the new class during the learning process. To address these challenges, we propose a novel framework, Incremental Sequential Labeling without Semantic Shifts (IS3). Motivated by the identified semantic shifts (E2O and O2E), IS3 aims to mitigate catastrophic forgetting in models. As for the E2O problem, we use knowledge distillation to maintain the model's discriminative ability for old entities. Simultaneously, to tackle the O2E problem, we alleviate the model's bias toward new entities through debiased loss and optimization levels. Our experimental evaluation, conducted on three datasets with various incremental settings, demonstrates the superior performance of IS3 compared to the previous state-of-the-art method by a significant margin. The data, code, and scripts are publicly available 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Learning "O" Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER cites Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic Samples
  • Prototypical Replay with Old-class Focusing Knowledge Distillation for Incremental Named Entity Recognition cites Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition
  • Prototypical Replay with Old-class Focusing Knowledge Distillation for Incremental Named Entity Recognition cites Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic Samples
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Prototypical Replay with Old-class Focusing Knowledge Distillation for Incremental Named Entity RecognitionZesheng Liu, Qiannan Zhu, Cuiping Li, Hong ChenAAAI 2025

Builds on11

  1. MAVEN: A Massive General Domain Event Detection DatasetXiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang et al.EMNLP 2020 · 143 citations
  2. Incremental Event Detection via Knowledge Consolidation NetworksPengfei Cao, Yubo Chen, Jun Zhao, Taifeng WangEMNLP 2020 · 41 citations
  3. Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic SamplesYu Xia, Quan Wang, Yajuan Lyu, Yong Zhu et al.ACL Findings 2022 · 32 citations
  4. Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity RecognitionJunhao Zheng, Zhanxian Liang, Haibin Chen, Qianli MaEMNLP 2022 · 20 citations
  5. Learning "O" Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NERRuotian Ma, Xuanting Chen, Zhang Lin, Xin Zhou et al.ACL 2023 · 12 citations

Related papers

  1. Few-Shot Class-Incremental Learning for Named Entity RecognitionRui Wang, Tong Yu, Handong Zhao, Sungchul Kim et al.ACL 2022 · 26 citations
  2. Modeling the Background for Incremental Learning in Semantic SegmentationFabio Cermelli, Massimiliano Mancini, Samuel Rota Bulò, Elisa Ricci et al.CVPR 2020
  3. Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental LearningRun He, Di Fang, Yicheng Xu, Yawen Cui et al.ICML 2025
  4. Incrementer: Transformer for Class-Incremental Semantic Segmentation with Knowledge Distillation Focusing on Old ClassChao Shang, Hongliang Li, Fanman Meng, Qingbo Wu et al.CVPR 2023
  5. Bridging Non Co-occurrence with Unlabeled In-the-wild Data for Incremental Object DetectionNa Dong, Yongqiang Zhang, Mingli Ding, Gim Hee LeeNeurIPS 2021 · 33 citations
Dusk over the sea between two cliffs drawn in fine vertical lines