Bayesian Sparsification of Deep C-valued Networks

With continual miniaturization ever more applications of deep learning can be found in embedded systems, where it is common to encounter data with natural representation in the complex domain. To this end we extend Sparse Variational Dropout to complex-valued neural networks and verify the proposed Bayesian technique by conducting a large numerical study of the performancecompression trade-off of C-valued networks on two tasks: image recognition on MNIST-like and CIFAR10 datasets and music transcription on Mu-sicNet. We replicate the state-of-the-art result by Trabelsi et al. (2018) on MusicNet with a complexvalued network compressed by 50 -100× at a small performance penalty.