ACL Findings2023
History repeats: Overcoming catastrophic forgetting for event-centric temporal knowledge graph completion
Abstract
Temporal knowledge graph (TKG) completion models typically rely on having access to the entire graph during training. However, in real-world scenarios, TKG data is often received incrementally as events unfold, leading to a dynamic non-stationary data distribution over time. While one could incorporate fine-tuning to existing methods to allow them to adapt to evolving TKG data, this can lead to forgetting previously learned patterns. Alternatively, retraining the model with the entire updated TKG can mitigate forgetting but is computationally burdensome. To address these challenges, we propose a general continual training framework that is applicable to any TKG completion method, and leverages two key ideas: (i) a temporal regularization that encourages repurposing of less important model parameters for learning new knowledge, and (ii) a clustering-based experience replay that reinforces the past knowledge by selectively preserving only a small portion of the past data. Our experimental results on widely used event-centric TKG datasets demonstrate the effectiveness of our proposed continual training framework in adapting to new events while reducing catastrophic forgetting. Further, we perform ablation studies to show the effectiveness of each component of our proposed framework. Finally, we investigate the relation between the memory dedicated to experience replay and the benefit gained from our clustering-based sampling strategy.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning cites Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks
- Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning cites Recurrent Event Network: Autoregressive Structure Inferenceover Temporal Knowledge Graphs
- Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning cites Diachronic Embedding for Temporal Knowledge Graph Completion
- Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks cites Diachronic Embedding for Temporal Knowledge Graph Completion
- Mining the Past with Dual Criteria: Integrating Three types of Historical Information for Context-aware Event Forecasting cites Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning
- A Generative Adaptive Replay Continual Learning Model for Temporal Knowledge Graph Reasoning cites Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning
- A Generative Adaptive Replay Continual Learning Model for Temporal Knowledge Graph Reasoning cites Recurrent Event Network: Autoregressive Structure Inferenceover Temporal Knowledge Graphs
- A Generative Adaptive Replay Continual Learning Model for Temporal Knowledge Graph Reasoning cites Diachronic Embedding for Temporal Knowledge Graph Completion
- Recurrent Event Network: Autoregressive Structure Inferenceover Temporal Knowledge Graphs cites Diachronic Embedding for Temporal Knowledge Graph Completion
Cited by top-tier papers2
- A Generative Adaptive Replay Continual Learning Model for Temporal Knowledge Graph ReasoningZhiyu Zhang, Wei Chen, Youfang Lin, Huaiyu WanACL 2025 · 4 citations
- Mining the Past with Dual Criteria: Integrating Three types of Historical Information for Context-aware Event ForecastingRong Ma, Lei Wang, Yating Yang, Bo Ma et al.EMNLP 2025
Builds on14
- Diachronic Embedding for Temporal Knowledge Graph CompletionRishab Goel, Seyed Mehran Kazemi, Marcus A. Brubaker, Pascal PoupartAAAI 2020 · 423 citations
- Recurrent Event Network: Autoregressive Structure Inferenceover Temporal Knowledge GraphsWoojeong Jin, Meng Qu, Xisen Jin, Xiang RenEMNLP 2020 · 353 citations
- Temporal Knowledge Graph Reasoning Based on Evolutional Representation LearningZixuan Li, Xiaolong Jin, Wei Li, Saiping Guan et al.SIGIR 2021 · 345 citations
- Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation NetworksCunchao Zhu, Muhao Chen, Changjun Fan, Guangquan Cheng et al.AAAI 2021 · 343 citations
- Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience ReplayFan Zhou, Chengtai CaoAAAI 2021 · 175 citations
Related papers
- TIE: A Framework for Embedding-based Incremental Temporal Knowledge Graph CompletionJiapeng Wu, Yishi Xu, Yingxue Zhang, Chen Ma et al.SIGIR 2021 · 27 citations
- Learning to Evolve: Bayesian-Guided Continual Knowledge Graph EmbeddingLinYu Li, Zhi Jin, Yuanpeng He, Dongming Jin et al.WWW 2026 · 1 citation
- Self-Supervised Continual Graph Learning via Adaptive Spaced Replay on Node ProxiesZhen Peng, Xu Hua, Jingchen Hao, Qika Lin et al.KDD 2025
- Towards Multimodal Continual Knowledge Embedding wth Modality Forgetting ModulationXiaowen Jiang, Jing Yang, Shundong Yang, Yuan Gao et al.AAAI 2026
- Streaming Graph Neural Networks with Generative ReplayJunshan Wang, Wenhao Zhu, Guojie Song, Liang WangKDD 2022 · 33 citations
