Improved Balanced Classification with Theoretically Grounded Loss Functions
Corinna Cortes, Mehryar Mohri, Yutao Zhong
Abstract
The balanced loss is a widely adopted objective for multi-class classification under class imbalance. By assigning equal importance to all classes, regardless of their frequency, it promotes fairness and ensures that minority classes are not overlooked. However, directly minimizing the balanced classification loss is typically intractable, which makes the design of effective surrogate losses a central question. This paper introduces and studies two advanced surrogate loss families: Generalized Logit-Adjusted (GLA) loss functions and Generalized Class-Aware weighted (GCA) losses. GLA losses generalize Logit-Adjusted losses, which shift logits based on class priors, to the broader general cross-entropy loss family. GCA loss functions extend the standard class-weighted losses, which scale losses inversely by class frequency, by incorporating class-dependent confidence margins and extending them to the general cross-entropy family. We present a comprehensive theoretical analysis of consistency for both loss families. We show that GLA losses are Bayes-consistent, but only H-consistent for complete (i.e., unbounded) hypothesis sets. Moreover, their H-consistency bounds depend inversely on the minimum class probability, scaling at least as 1/p min . In contrast, GCA losses are H-consistent for any hypothesis set that is bounded or complete, with H-consistency bounds that scale more favorably as 1/ √ p min , offering significantly stronger theoretical guarantees in imbalanced settings. We report the results of experiments demonstrating that, empirically, both the GCA losses with calibrated class-dependent confidence margins and GLA losses can greatly outperform straightforward class-weighted losses as well as the LA losses. GLA generally performs slightly better in common benchmarks, whereas GCA exhibits a slight edge in highly imbalanced settings. Thus, we advocate for both GLA and GCA losses as principled, theoretically sound, and state-of-the-art surrogates for balanced classification under class imbalance.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
6 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Experts cites Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defer
- Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Experts cites Cross-Entropy Loss Functions: Theoretical Analysis and Applications
- Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Experts cites Mind the Gap: Structure-Aware Consistency in Preference Learning
- Mind the Gap: Structure-Aware Consistency in Preference Learning cites Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Experts
- Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Experts cites Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction
- Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction cites Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Experts
- Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Experts cites Optimized Deferral for Imbalanced Settings
- Optimized Deferral for Imbalanced Settings cites Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k Experts
- Long-tail learning via logit adjustment cites Decoupling Representation and Classifier for Long-Tailed Recognition
- Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defer cites Cross-Entropy Loss Functions: Theoretical Analysis and Applications
- Mind the Gap: Structure-Aware Consistency in Preference Learning cites Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defer
- Mind the Gap: Structure-Aware Consistency in Preference Learning cites Cross-Entropy Loss Functions: Theoretical Analysis and Applications
- Mind the Gap: Structure-Aware Consistency in Preference Learning cites Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction
- Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction cites Mind the Gap: Structure-Aware Consistency in Preference Learning
- Mind the Gap: Structure-Aware Consistency in Preference Learning cites Optimized Deferral for Imbalanced Settings
- Optimized Deferral for Imbalanced Settings cites Mind the Gap: Structure-Aware Consistency in Preference Learning
- Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction cites Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defer
- Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction cites Cross-Entropy Loss Functions: Theoretical Analysis and Applications
- Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction cites Optimized Deferral for Imbalanced Settings
- Optimized Deferral for Imbalanced Settings cites Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction
- Balanced Meta-Softmax for Long-Tailed Visual Recognition cites Decoupling Representation and Classifier for Long-Tailed Recognition
- Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect cites Decoupling Representation and Classifier for Long-Tailed Recognition
- Optimized Deferral for Imbalanced Settings cites Long-tail learning via logit adjustment
- Optimized Deferral for Imbalanced Settings cites Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defer
- Optimized Deferral for Imbalanced Settings cites Cross-Entropy Loss Functions: Theoretical Analysis and Applications
- Optimized Deferral for Imbalanced Settings cites Balanced Meta-Softmax for Long-Tailed Visual Recognition
- Optimized Deferral for Imbalanced Settings cites Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect
- Optimized Deferral for Imbalanced Settings cites Decoupling Representation and Classifier for Long-Tailed Recognition
Cited by top-tier papers6
- Why Ask One When You Can Ask k? Learning-to-Defer to the Top-k ExpertsYannis Montreuil, Axel Carlier, Lai Xing Ng, Wei Tsang OoiICLR 2026 · 7 citations
- Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured PredictionMehryar Mohri, Yutao ZhongICML 2026 · 7 citations
- Optimized Deferral for Imbalanced SettingsCorinna Cortes, Anqi Mao, Mehryar Mohri, Yutao ZhongICML 2026 · 7 citations
- Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to DeferAnqi Mao, Mehryar Mohri, Yutao ZhongICML 2025
- Mind the Gap: Structure-Aware Consistency in Preference LearningMehryar Mohri, Yutao ZhongICML 2026
Builds on57
- Decoupling Representation and Classifier for Long-Tailed RecognitionBingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan et al.ICLR 2020 · 1,496 citations
- Long-tail learning via logit adjustmentAditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain et al.ICLR 2021 · 937 citations
- Balanced Meta-Softmax for Long-Tailed Visual RecognitionJiawei Ren, Cunjun Yu, Shunan Sheng, Xiao Ma et al.NeurIPS 2020 · 861 citations
- Cross-Entropy Loss Functions: Theoretical Analysis and ApplicationsAnqi Mao, Mehryar Mohri, Yutao ZhongICML 2023 · 790 citations
- Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal EffectKaihua Tang, Jianqiang Huang, Hanwang ZhangNeurIPS 2020 · 533 citations
Related papers
- Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced DataCorinna Cortes, Anqi Mao, Mehryar Mohri, Yutao ZhongICML 2025
- Multi-Class -Consistency BoundsPranjal Awasthi, Anqi Mao, Mehryar Mohri, Yutao ZhongNeurIPS 2022 · 48 citations
- Multi-Label Learning with Stronger Consistency GuaranteesAnqi Mao, Mehryar Mohri, Yutao ZhongNeurIPS 2024 · 30 citations
- Class and Attribute-Aware Logit Adjustment for Generalized Long-Tail LearningXiaoling Zhou, Ou Wu, Nan YangAAAI 2025 · 5 citations
- H-Consistency Bounds: Characterization and ExtensionsAnqi Mao, Mehryar Mohri, Yutao ZhongNeurIPS 2023 · 34 citations
