Lune

ICML2021顶会

Overcoming Catastrophic Forgetting by Bayesian Generative Regularization

Pei-Hung Chen, Wei Wei, Cho-Jui Hsieh, Bo Dai

摘要

The streaming update of Bayesian posterior calculation provides us a natural way for continual learning. However, the naïve mean-field posterior parametrization for variational approximation is inappropiate in neural network, and thus, lock the full ability for preventing catastrophic forgetting. To resolve this issue, we introduce a generative regularization for all given classification models, which is implemented by leveraging energy-based models with contrastive loss, to obtain the sufficient features for valid decomposition in posterior approxiamtion. By combining discriminative and generative loss together, we empirically show that the proposed method outperforms state-ofthe-art methods on a variety of tasks, avoiding catastrophic forgetting in continual learning. In particular, the proposed method outperforms baseline methods over 15% on the Fashion-MNIST dataset and 10% on the CUB dataset.

问 Lune

几分钟读透这篇 Paper,不用再花一下午。

Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。

已有 1 篇顶会 Paper 引用了它。别等审稿人问起,先弄清它们发现了什么。

可以从这些问题问起

Lune 可以查阅

全文 · 引用它的顶会 Paper · 相关的 Paper

获取答案

免费开始,无需绑卡

脉络

它引用的引用它的从左到右按年份排列,引用越多,位置越高。

引用它的顶会 Paper1

  1. Do Bayesian Neural Networks Actually Behave Like Bayesian Models?Gábor Pituk, Vik Shirvaikar, Tom RainforthICML 2025

它引用的 Paper2

  1. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 次引用
  2. Your classifier is secretly an energy based model and you should treat it like oneWill Grathwohl, Kuan-Chieh Wang, Jörn-Henrik Jacobsen, David Duvenaud 等ICLR 2020 · 643 次引用

相关 Paper

  1. BooVAE: Boosting Approach for Continual Learning of VAEEvgenii Egorov, Anna Kuzina, Evgeny BurnaevNeurIPS 2021 · 34 次引用
  2. Bayesian Structural Adaptation for Continual LearningAbhishek Kumar, Sunabha Chatterjee, Piyush RaiICML 2021 · 7 次引用
  3. Uncertainty-guided Continual Learning with Bayesian Neural NetworksSayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, Marcus RohrbachICLR 2020 · 211 次引用
  4. Temporal-Difference Variational Continual LearningLuckeciano Carvalho Melo, Alessandro Abate, Yarin GalNeurIPS 2025 · 1 次引用
  5. Variational Auto-Regressive Gaussian Processes for Continual LearningSanyam Kapoor, Theofanis Karaletsos, Thang D. BuiICML 2021 · 32 次引用
黄昏的海面,两侧是细线勾勒的悬崖