Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models

Incremental Learning (IL) has been a longstanding problem in both vision and Natural Language Processing (NLP) communities. In recent years, as Pre-trained Language Models (PLMs) have achieved remarkable progress in various NLP downstream tasks, utilizing PLMs as backbones has become a common practice in recent research of IL in NLP. Most assume that catastrophic forgetting is the biggest obstacle to achieving superior IL performance and propose various techniques to overcome this issue. However, we find that this assumption is problematic. Specifically, we revisit more than 20 methods on four classification tasks (Text Classification, Intent Classification, Relation Extraction, and Named Entity Recognition) under the two most popular IL settings (Class-Incremental and Task-Incremental) and reveal that most of them severely underestimate the inherent anti-forgetting ability of PLMs. Based on the observation, we propose a frustratingly easy method called SEQ* for IL with PLMs. The results show that SEQ* has competitive or superior performance compared with state-ofthe-art (SOTA) IL methods yet requires considerably less trainable parameters and training time. These findings urge us to revisit the IL with PLMs and encourage future studies to have a fundamental understanding of the catastrophic forgetting in PLMs. The data, code and scripts are in the supplementray material and will be publicly available 1 . Incremental Learning (IL) and has been impeded 040 by catastrophic forgetting (Kirkpatrick et al., 2017). 041 Catastrophic forgetting refers to neural networks 042 forgetting previous knowledge after learning new 043 tasks (McCloskey and Cohen, 1989). 044 Recent years have witnessed significant break-045 throughs in Pre-trained Language Models (PLMs) 046 in vision and NLP tasks. Most recent studies of IL 047 use PLMs as the backbone and design various meth-048 ods for alleviating catastrophic forgetting in NLP 049 tasks. However, is forgetting really catastrophical 050 in PLMs? More specifically, how can we quantify 051 forgetting and how much knowledge is forgotten in 052 various IL scenarios when using various backbones 053 and methods on various tasks? 054 To answer the above question, we carry out 055 extensive experiments to explore forgetting in 056 more than 20 methods on four classification tasks 057 (Text Classification, Intent Classification, Rela-058 tion Extraction, and Named Entity Recognition) 059 1 under the two most popular IL settings (Class-060 Incremental and Task-Incremental) with various 061 model architecture (encoder only and decoder only) 062 and scales (from 19M to 1.21B number of param-063 eters). Through extensive experiments, we have 064 several major findings: 065 • The popular assumption that PLMs suffer 066 from catastrophic forgetting does not hold. 067 Even under sequential fine-tuning (SEQ), the 068 PLMs maintain the knowledge without much 069 forgetting (Sec. 3.2). From the probing per-070 spective, most existing IL methods do not 071 learn incremental knowledge for PLMs (Sec. 072 4.2). 073 • By combining SEQ with simple strategies 074 (Sec. 4.1), we propose SEQ* and find that 075 SEQ* has competitive or even superior perfor-076 mance than SOTA IL methods (Figure 1, Sec. 077 4.2). 078 • The inherent anti-forgetting ability of PLMs 079 comes from both the pre-training stage as well 080 as the architecture of Transformer (Sec. 3.4). 081 Randomly initialised PLMs learn incremen-082 tally when SEQ is performed on a sequence 083 of tasks. 084 • The forgetting of SEQ is due to the deviation 085 of the classifier from the PLM rather than the 086 loss of old knowledge in the PLM. (Sec. 3.5). 087 Our study urges the NLP community to revisit 088 and deepen the understanding of the forgetting in 089 PLMs. 090 2 Experimental Settings 091 2.1 Problem formulation 092 Formally, the goal of IL is to learn a model 093 f θ : x → y ∈ Y from a sequence of tasks 094 where the t-th task D t = 095 (x t i , y t i ) i=1 contains input samples x t i ∈ X t and 096 labels y t i ∈ Y t . In Class-Incremental Learning 097 (CIL), the label sets of different tasks are exclusive: 098 and the task id is unknown 099 during inference. In Task-Incremental Learning 100 (TIL), the label sets of different tasks may be over-101 lapping: Y 1 ∩ Y 2 • • • Y T ̸ = ∅, and the task id is 102 required during inference. In general, CIL is much 103 more challenging than TIL because PLMs suffer 104 from inter-task forgetting much more seriously than 105 intra-task forgetting (Tao et al., 2023a). Appendix 106 188 Lower Bound? 189 Sequential fine-tuning (SEQ) has long been re-190 garded as the lower bound of IL. In this subsection, 191 we revisit SEQ from the probing perspective, and 192 we find that SEQ is severely underestimated when 193 using PLMs for IL. 194 The backbone was small and randomly initial-195