Beyond Smoothness: Incorporating Low-Rank Analysis into Nonparametric Density Estimation

The construction and theoretical analysis of the most popular universally consistent nonparametric density estimators hinge on one functional property: smoothness. In this paper we investigate the theoretical implications of incorporating a multi-view latent variable model, a type of low-rank model, into nonparametric density estimation. To do this we perform extensive analysis on histogram-style estimators that integrate a multi-view model. Our analysis culminates in showing that there exists a universally consistent histogram-style estimator that converges to any multi-view model with a finite number of Lipschitz continuous components at a rate of O(1/ 3 √ n) in L 1 error. In contrast, the standard histogram estimator can converge at a rate slower than 1/ d √ n on the same class of densities. We also introduce a new nonparametric latent variable model based on the Tucker decomposition. A rudimentary implementation of our estimators experimentally demonstrates a considerable performance improvement over the standard histogram estimator. We also provide a thorough analysis of the sample complexity of our Tucker decomposition-based model and a variety of other results. Thus, our paper provides solid theoretical foundations for extending low-rank techniques to the nonparametric setting. * For an estimator V restricted space a of densities P, the estimation error refers to the difference between V -p and minq∈P p -q , where p is the target density. This is similar to estimator variance. † A density estimator is universally consistent if it asymptotically recovers any density.