Lune

ACL Findings2024

An Ensemble-of-Experts Framework for Rehearsal-free Continual Relation Extraction

Shen Zhou, Yongqi Li, Xin Miao, Tieyun Qian

Abstract

Continual relation extraction (CRE) aims to continuously learn relations in new tasks without forgetting old relations in previous tasks. Current CRE methods are all rehearsal-based, which need to store samples and thus may encounter privacy and security issues. This paper targets rehearsal-free continual relation extraction for the first time and decomposes it into task identification and within-task prediction sub-problems. Existing rehearsal-free methods focus on training a model (expert) for withintask prediction yet neglect to enhance the models' capability of task identification.

In this paper, we propose an Ensemble-of-Experts (EoE) framework for rehearsal-free continual relation extraction. Specifically, we first discriminatively train each expert by augmenting analogous relations across tasks to enhance the expert's task identification ability. We then propose a cascade voting mechanism to form an ensemble of experts for effectively aggregating their abilities. Extensive experiments show that our method outperforms current rehearsal-free methods and is even better than rehearsal-based CRE methods. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Generative Meta-Learning for Zero-Shot Relation Triplet ExtractionWanli Li, Tieyun Qian, Yi Song, Zeyu Zhang et al.SIGIR 2025 · 4 citations

Builds on17

  1. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  2. Learning to Prompt for Continual LearningZifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang et al.CVPR 2022 · 635 citations
  3. Learning from Context or Names? An Empirical Study on Neural Relation ExtractionHao Peng, Tianyu Gao, Xu Han, Yankai Lin et al.EMNLP 2020 · 185 citations
  4. A Theoretical Study on Solving Continual LearningGyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke et al.NeurIPS 2022 · 119 citations
  5. Continual Relation Learning via Episodic Memory Activation and ReconsolidationXu Han, Yi Dai, Tianyu Gao, Yankai Lin et al.ACL 2020 · 92 citations

Related papers

  1. Continual Relation Extraction via Sequential Multi-Task LearningThanh-Thien Le, Manh Nguyen, Tung Thanh Nguyen, Ngo Van Linh et al.AAAI 2024 · 16 citations
  2. Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation ExtractionLi Cui, Deqing Yang, Jiaxin Yu, Chengwei Hu et al.ACL 2021
  3. Improving Continual Relation Extraction by Distinguishing Analogous SemanticsWenzheng Zhao, Yuanning Cui, Wei HuACL 2023 · 19 citations
  4. Consistent Representation Learning for Continual Relation ExtractionKang Zhao, Hua Xu, Jiangong Yang, Kai GaoACL Findings 2022 · 45 citations
  5. Consistent Prototype Learning for Few-Shot Continual Relation ExtractionXiudi Chen, Hui Wu, Xiaodong ShiACL 2023 · 17 citations
Dusk over the sea between two cliffs drawn in fine vertical lines