coVariance Neural Networks

Learning deep representations from covariance in-formation via coVariance Neural Networks (VNNs) has shown an improved performance and insights with respect to Principal Component Analysis (PCA)-based alternatives and better stability in finite-sample regimes. VNNs extend the PCA transform by learning end-to-end the spectral processing function on the principal directions of the data in each layer. However, VNNs operate on the pre-computed sample covariance matrix, which is prone to estimation errors, sensitive to outliers, and not adapted to the task at hand. To overcome this limitation, we propose Robust coVariance Neural Networks (RVNNs), a framework that simultaneously learns a robust estimator of the covariance matrix and the VNN parameters in an end-to-end manner, leading to a fully task-aware pipeline. We prove that RVNNs combine the robustness to outliers with the finite-sample stability of VNNs, and we show that their end-to-end robust covariance learning leads to better prediction performance compared to robust PCA-based approaches on simulated and real-world data from brain recordings and human motion sensor measurements.