CVPR Findings2026
Rich Feature Learning via Diversification
Xi Leng, Yongqiang Chen, Xiaoying Tang, Yatao Bian
Abstract
Rich Feature Learning (RFL) aims to extract all beneficial features from the training distribution and shows promise for Out-of-Distribution (OOD) generalization. Despite its success, a precise and comprehensive definition of "richness" remains elusive. Through an in-depth comparison between RFL and empirical risk minimization (ERM), we identify that feature diversity is the key differentiator driving RFL's superior OOD performance. Building on this insight, we contribute a formal definition of rich features, encompassing both informativeness and diversity. Leveraging this foundation, we propose Diversity-fOunded Rich fEature lEarniNg (DOREEN), a simple yet highly effective RFL algorithm that trains multiple models with identical architectures concurrently to promote feature diversity. We theoretically demonstrate that DOREEN not only realizes the benefits of RFL but also addresses the limitations of prior RFL algorithms. Extensive experiments validate that DOREEN learns richer features and consistently enhances OOD performance across various OOD objectives.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Out-of-Distribution Generalization via Risk Extrapolation (REx) cites In Search of Lost Domain Generalization
- Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization cites Out-of-Distribution Generalization via Risk Extrapolation (REx)
- Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization cites Distributionally Robust Neural Networks
- Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization cites In Search of Lost Domain Generalization
Cited by top-tier papers1
Builds on26
- WILDS: A Benchmark of in-the-Wild Distribution ShiftsPang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie et al.ICML 2021 · 1,773 citations
- Distributionally Robust Neural NetworksShiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, Percy LiangICLR 2020 · 1,578 citations
- In Search of Lost Domain GeneralizationIshaan Gulrajani, David Lopez-PazICLR 2021 · 1,416 citations
- Out-of-Distribution Generalization via Risk Extrapolation (REx)David Krueger, Ethan Caballero, Jörn-Henrik Jacobsen, Amy Zhang et al.ICML 2021 · 1,163 citations
- Invariance Principle Meets Information Bottleneck for Out-of-Distribution GeneralizationKartik Ahuja, Ethan Caballero, Dinghuai Zhang, Jean-Christophe Gagnon-Audet et al.NeurIPS 2021 · 372 citations
Related papers
- Understanding and Improving Feature Learning for Out-of-Distribution GeneralizationYongqiang Chen, Wei Huang, Kaiwen Zhou, Yatao Bian et al.NeurIPS 2023 · 49 citations
- On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution GeneralizationShiji Xin, Yifei Wang, Jingtong Su, Yisen WangAAAI 2023 · 14 citations
- An Empirical Investigation of Domain Generalization with Empirical Risk MinimizersRamakrishna Vedantam, David Lopez-Paz, David J. SchwabNeurIPS 2021 · 49 citations
- Sparse Invariant Risk MinimizationXiao Zhou, Yong Lin, Weizhong Zhang, Tong ZhangICML 2022 · 85 citations
- Generalization Bounds for Out-of-distribution GeneralizationXin Zou, Xiuwen Gong, Weiwei LiuICML 2026 · 14 citations
