Decouple Your Discovery and Memory in Continual Generalized Category Discovery

Continual Generalized Category Discovery (C-GCD) seeks to incrementally discover new categories from unlabeled data and memorize old categories' knowledge, fostering model adaptability in real-world scenarios. Recent efforts focus on devising specific frameworks and various anti-forgetting strategies, striving for a typical stabilityplasticity trade-off. In this work, we first revisit previous methods and identify that most of them over-protect old classes, hampering the accurate discovery of novel ones. To address this challenge, we introduce the Decouple Your Discovery and Memory (DYDM), a dual-branch architecture that decouples the discovery of new classes and the memorization of old classes. The discovery branch is focused on accurately recognizing new classes, while the memory branch consolidates all identified categories in a recursive manner and functions as the inference branch. Importantly, benefiting from the strong knowledge retention ability of the memory branch, the discovery branch can facilitate the recognition of novel classes from the unlabeled data, achieving a win-win outcome between plasticity and stability. Extensive experiments on various datasets and settings demonstrate the superiority of our approach. Furthermore, our framework can integrate with existing approaches, consistently enhancing their performance.