Universality laws for Gaussian mixtures in generalized linear models

Let (x i , y i ) i=1,...,n denote independent samples from a general mixture distribution c∈C ρ c P x c , and consider the hypothesis class of generalized linear models ŷ = F (Θ x). In this work, we investigate the asymptotic joint statistics of the family of generalized linear estimators (Θ 1 , . . . , Θ M ) obtained either from (a) minimizing an empirical risk Rn (Θ; X, y) or (b) sampling from the associated Gibbs measure exp(-βn Rn (Θ; X, y)). Our main contribution is to characterize under which conditions the asymptotic joint statistics of this family depends (on a weak sense) only on the means and covariances of the class conditional features distribution P x c . In particular, this allow us to prove the universality of different quantities of interest, such as the training and generalization errors, redeeming a recent line of work in high-dimensional statistics working under the Gaussian mixture hypothesis. Finally, we discuss the applications of our results to different machine learning tasks of interest, such as ensembling and uncertainty quantification.