Space Alignment Matters: The Missing Piece for Inducing Neural Collapse in Long-Tailed Learning
Jinping Wang, Zhiqiang Gao, Zhiwu Xie
Abstract
Recent studies on Neural Collapse (NC) reveal that, under class-balanced conditions, the class feature means and the classifier weights spontaneously align into a simplex equiangular tight frame (ETF). In long-tailed regimes, however, severe sample imbalance tends to prevent the emergence of the NC phenomenon, resulting in poor generalization performance.Current efforts predominantly seek to recover the ETF geometry by imposing constraints on features or classifier weights, yet overlook a critical problem: There is a pronounced misalignment between the feature and the classifier weight spaces. In this paper, we theoretically quantify the harm of such misalignment through an optimal error exponent analysis.Built on this insight, we propose three explicit alignment strategies that plug-and-play into existing long-tail methods without architectural change. Extensive experiments on the CIFAR-10-LT, CIFAR-100-LT, and ImageNet-LT datasets consistently boost examined baselines and achieve the state-of-the-art performances.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 8c0f51b8-0fc1-4d2c-8a57-ce0b3741eafeCited by top-tier papers1
Ask how each one uses itBuilds on11
- Long-tailed Recognition by Routing Diverse Distribution-Aware ExpertsXudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu et al.ICLR 2021 · 481 citations
- Exploring Balanced Feature Spaces for Representation LearningBingyi Kang, Yu Li, Sa Xie, Zehuan Yuan et al.ICLR 2021 · 296 citations
- Targeted Supervised Contrastive Learning for Long-Tailed RecognitionTianhong Li, Peng Cao, Yuan Yuan, Lijie Fan et al.CVPR 2022 · 196 citations
- Balanced Contrastive Learning for Long-Tailed Visual RecognitionJianggang Zhu, Zheng Wang, Jingjing Chen, Yi-Ping Phoebe Chen et al.CVPR 2022 · 194 citations
- Long- Tailed Recognition via Weight BalancingShaden Alshammari, Yu-Xiong Wang, Deva Ramanan, Shu KongCVPR 2022 · 133 citations
Related papers
- MLC-NC: Long-Tailed Multi-Label Image Classification Through the Lens of Neural CollapseZijian Tao, Shao-Yuan Li, Wenhai Wan, Jinpeng Zheng et al.AAAI 2025 · 7 citations
- Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Features ModelHien Dang, Tho Tran Huu, Tan Minh Nguyen, Nhat HoICML 2024 · 19 citations
- Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced DataHien Dang, Tho Tran Huu, Stanley J. Osher, Hung Tran-The et al.ICML 2023 · 44 citations
- Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class-Incremental LearningYibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin et al.ICLR 2023 · 22 citations
- Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?Yibo Yang, Shixiang Chen, Xiangtai Li, Liang Xie et al.NeurIPS 2022 · 144 citations
