Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment

Real-world data distributions are often highly skewed. This has spurred a growing body of research on long-tailed recognition, aimed at addressing the imbalance in training classification models. Among the methods studied, multiplicative logit adjustment (MLA) stands out as a simple and effective method. What theoretical foundation explains the effectiveness of this heuristic method? We provide a justification for the effectiveness of MLA with the following two-step process. First, we develop a theory that adjusts optimal decision boundaries by estimating feature spread on the basis of neural collapse. Second, we demonstrate that MLA approximates this optimal method. Additionally, through experiments on long-tailed datasets, we illustrate the practical usefulness of MLA under more realistic conditions. We also offer experimental insights to guide the tuning of MLA hyperparameters.

Published as a conference paper at ICLR 2025

• We derive a theoretical framework for adjusting decision boundaries optimally by using feature spread estimates from neural collapse (NC) (Papyan et al., 2020) (Section 4.2).

• We demonstrate that MLA is effective for LTR by proving it has similar decision boundaries to the method based on the aforementioned theory (Section 4.3). This clarifies under what conditions this approximation holds and how adjustments should be made.

• We experimentally validate that the approximation of MLA holds under realistic, non-ideal conditions where NC is not fully realized (Section 5).

• We provide empirical guidelines for hyperparameter-tuning of MLA (Section 5.4).