HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts
Abstract
Generalized Category Discovery (GCD) is a challenging task in which, given a partially labelled dataset, models must categorize all unlabelled instances, regardless of whether they come from labelled categories or from new ones. In this paper, we challenge a remaining assumption in this task: that all images share the same domain. Specifically, we introduce a new task and method to handle GCD when the unlabelled data also contains images from different domains to the labelled set. Our proposed 'HiLo' networks extract High-level semantic and Low-level domain features, before minimizing the mutual information between the representations. Our intuition is that the clusterings based on domain information and semantic information should be independent. We further extend our method with a specialized domain augmentation tailored for the GCD task, as well as a curriculum learning approach. Finally, we construct a benchmark from corrupted fine-grained datasets as well as a large-scale evaluation on DomainNet with real-world domain shifts, reimplementing a number of GCD baselines in this setting. We demonstrate that HiLo outperforms SoTA category discovery models by a large margin on all evaluations.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
13 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Emerging Properties in Self-Supervised Vision Transformers cites A Simple Framework for Contrastive Learning of Visual Representations
- Emerging Properties in Self-Supervised Vision Transformers cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery cites Emerging Properties in Self-Supervised Vision Transformers
- SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- HIDISC: A Hyperbolic Framework for Domain Generalization with Generalized Category Discovery cites Emerging Properties in Self-Supervised Vision Transformers
- HIDISC: A Hyperbolic Framework for Domain Generalization with Generalized Category Discovery cites CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
- HIDISC: A Hyperbolic Framework for Domain Generalization with Generalized Category Discovery cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale cites A Simple Framework for Contrastive Learning of Visual Representations
- TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery cites Emerging Properties in Self-Supervised Vision Transformers
- TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery cites Learning Transferable Visual Models From Natural Language Supervision
- Learning Transferable Visual Models From Natural Language Supervision cites A Simple Framework for Contrastive Learning of Visual Representations
- Learning Transferable Visual Models From Natural Language Supervision cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective cites Emerging Properties in Self-Supervised Vision Transformers
- Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective cites Learning Transferable Visual Models From Natural Language Supervision
- Open-World Deepfake Attribution via Confidence-Aware Asymmetric Learning cites Emerging Properties in Self-Supervised Vision Transformers
Cited by top-tier papers13
- SEAL: Semantic-Aware Hierarchical Learning for Generalized Category DiscoveryZhenqi He, Yuanpei Liu, Kai HanNeurIPS 2025 · 10 citations
- Generalized Category Discovery under Domain Shift: A Frequency Domain PerspectiveWei Feng, Zongyuan GeNeurIPS 2025 · 9 citations
- Open-World Deepfake Attribution via Confidence-Aware Asymmetric LearningHaiyang Zheng, Nan Pu, Wenjing Li, Teng Long et al.AAAI 2026 · 5 citations
- HIDISC: A Hyperbolic Framework for Domain Generalization with Generalized Category DiscoveryVaibhav Rathore, Divyam Gupta, Biplab BanerjeeNeurIPS 2025 · 3 citations
- TALON: Test-time Adaptive Learning for On-the-Fly Category DiscoveryYanan Wu, Yuhan Yan, Tailai Chen, Zhixiang Chi et al.CVPR 2026 · 3 citations
Builds on53
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
- Emerging Properties in Self-Supervised Vision TransformersMathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou et al.ICCV 2021 · 8,921 citations
- CutMix: Regularization Strategy to Train Strong Classifiers With Localizable FeaturesSangdoo Yun, Dongyoon Han, Sanghyuk Chun, Seong Joon Oh et al.ICCV 2019 · 5,843 citations
Related papers
- DebGCD: Debiased Learning with Distribution Guidance for Generalized Category DiscoveryYuanpei Liu, Kai HanICLR 2025
- No Representation Rules Them All in Category DiscoverySagar Vaze, Andrea Vedaldi, Andrew ZissermanNeurIPS 2023 · 79 citations
- Incremental Generalized Category DiscoveryBingchen Zhao, Oisin Mac AodhaICCV 2023 · 36 citations
- Learning Semi-supervised Gaussian Mixture Models for Generalized Category DiscoveryBingchen Zhao, Xin Wen, Kai HanICCV 2023 · 109 citations
- MOS: Modeling Object-Scene Associations in Generalized Category DiscoveryZhengyuan Peng, Jinpeng Ma, Zhimin Sun, Ran Yi et al.CVPR 2025
