ACL Findings2024
An Ensemble-of-Experts Framework for Rehearsal-free Continual Relation Extraction
Abstract
Continual relation extraction (CRE) aims to continuously learn relations in new tasks without forgetting old relations in previous tasks. Current CRE methods are all rehearsal-based, which need to store samples and thus may encounter privacy and security issues. This paper targets rehearsal-free continual relation extraction for the first time and decomposes it into task identification and within-task prediction sub-problems. Existing rehearsal-free methods focus on training a model (expert) for withintask prediction yet neglect to enhance the models' capability of task identification.
In this paper, we propose an Ensemble-of-Experts (EoE) framework for rehearsal-free continual relation extraction. Specifically, we first discriminatively train each expert by augmenting analogous relations across tasks to enhance the expert's task identification ability. We then propose a cascade voting mechanism to form an ensemble of experts for effectively aggregating their abilities. Extensive experiments show that our method outperforms current rehearsal-free methods and is even better than rehearsal-based CRE methods. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
Cited by top-tier papers1
Builds on17
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Learning to Prompt for Continual LearningZifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang et al.CVPR 2022 · 635 citations
- Learning from Context or Names? An Empirical Study on Neural Relation ExtractionHao Peng, Tianyu Gao, Xu Han, Yankai Lin et al.EMNLP 2020 · 185 citations
- A Theoretical Study on Solving Continual LearningGyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke et al.NeurIPS 2022 · 119 citations
- Continual Relation Learning via Episodic Memory Activation and ReconsolidationXu Han, Yi Dai, Tianyu Gao, Yankai Lin et al.ACL 2020 · 92 citations
Related papers
- Continual Relation Extraction via Sequential Multi-Task LearningThanh-Thien Le, Manh Nguyen, Tung Thanh Nguyen, Ngo Van Linh et al.AAAI 2024 · 16 citations
- Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation ExtractionLi Cui, Deqing Yang, Jiaxin Yu, Chengwei Hu et al.ACL 2021
- Improving Continual Relation Extraction by Distinguishing Analogous SemanticsWenzheng Zhao, Yuanning Cui, Wei HuACL 2023 · 19 citations
- Consistent Representation Learning for Continual Relation ExtractionKang Zhao, Hua Xu, Jiangong Yang, Kai GaoACL Findings 2022 · 45 citations
- Consistent Prototype Learning for Few-Shot Continual Relation ExtractionXiudi Chen, Hui Wu, Xiaodong ShiACL 2023 · 17 citations
