Space Alignment Matters: The Missing Piece for Inducing Neural Collapse in Long-Tailed Learning
Jinping Wang, Zhiqiang Gao, Zhiwu Xie
摘要
Recent studies on Neural Collapse (NC) reveal that, under class-balanced conditions, the class feature means and the classifier weights spontaneously align into a simplex equiangular tight frame (ETF). In long-tailed regimes, however, severe sample imbalance tends to prevent the emergence of the NC phenomenon, resulting in poor generalization performance.Current efforts predominantly seek to recover the ETF geometry by imposing constraints on features or classifier weights, yet overlook a critical problem: There is a pronounced misalignment between the feature and the classifier weight spaces. In this paper, we theoretically quantify the harm of such misalignment through an optimal error exponent analysis.Built on this insight, we propose three explicit alignment strategies that plug-and-play into existing long-tail methods without architectural change. Extensive experiments on the CIFAR-10-LT, CIFAR-100-LT, and ImageNet-LT datasets consistently boost examined baselines and achieve the state-of-the-art performances.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
引用它的顶会 Paper1
问问它们各自怎么用它它引用的顶会 Paper11
- Long-tailed Recognition by Routing Diverse Distribution-Aware ExpertsXudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu 等ICLR 2021 · 被引用 481 次
- Exploring Balanced Feature Spaces for Representation LearningBingyi Kang, Yu Li, Sa Xie, Zehuan Yuan 等ICLR 2021 · 被引用 296 次
- Targeted Supervised Contrastive Learning for Long-Tailed RecognitionTianhong Li, Peng Cao, Yuan Yuan, Lijie Fan 等CVPR 2022 · 被引用 196 次
- Balanced Contrastive Learning for Long-Tailed Visual RecognitionJianggang Zhu, Zheng Wang, Jingjing Chen, Yi-Ping Phoebe Chen 等CVPR 2022 · 被引用 194 次
- Long- Tailed Recognition via Weight BalancingShaden Alshammari, Yu-Xiong Wang, Deva Ramanan, Shu KongCVPR 2022 · 被引用 133 次
相关 Paper
- MLC-NC: Long-Tailed Multi-Label Image Classification Through the Lens of Neural CollapseZijian Tao, Shao-Yuan Li, Wenhai Wan, Jinpeng Zheng 等AAAI 2025 · 被引用 7 次
- Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Features ModelHien Dang, Tho Tran Huu, Tan Minh Nguyen, Nhat HoICML 2024 · 被引用 19 次
- Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced DataHien Dang, Tho Tran Huu, Stanley J. Osher, Hung Tran-The 等ICML 2023 · 被引用 44 次
- Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class-Incremental LearningYibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin 等ICLR 2023 · 被引用 22 次
- Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?Yibo Yang, Shixiang Chen, Xiangtai Li, Liang Xie 等NeurIPS 2022 · 被引用 144 次
