Rationale-Enhanced Language Models are Better Continual Relation Learners

Continual relation extraction (CRE) aims to solve the problem of catastrophic forgetting when learning a sequence of newly emerging relations. Recent CRE studies have found that catastrophic forgetting arises from the model's lack of robustness against future analogous relations. To address the issue, we introduce rationale, i.e., the explanations of relation classification results generated by large language models (LLM), into CRE task. Specifically, we design the multi-task rationale tuning strategy to help the model learn current relations robustly. We also conduct contrastive rationale replay to further distinguish analogous relations. Experimental results on two standard benchmarks demonstrate that our method outperforms the state-of-the-art CRE models. Our code is available at https://github.com/WeiminXiong/ RationaleCL * Corresponding Author Similar relations: follows; followed by; characters Rationale: The relation between "Predator 2" (1990) and "Predators"(2010) is "followed by" because "Predator 2" is the second installment of the "Predator" film series, and it is followed by the fourth installment, which is "Predators". Prompt: The relation between "Predator 2" and "Predators" is "followed by", instead of "follows" or "characters". Please explain why. Contrastive Rationale: The sentence states that "Predator 2" took place before "Predators", so the proper relationship to denote this would be "followed by". The phrase "follows" suggests that "Predator 2" took place after "Predators". The phrase "characters" has no relation to the two films, as it refers to the people in a movie. Question: It is the fourth installment in the "Predator" film series, set to take place between "Predator 2" (1990) and "Predators" (2010). Classify the relation in above sentence. Prompt: The relation between "Predator 2" and "Predators" is "followed by". Please explain why.