Convolutional dictionary learning based auto-encoders for natural exponential-family distributions

We introduce a class of auto-encoder neural networks tailored to data fromnatural exponential family (e.g., count data). The architectures areby the problem of learning the filters in a convolutional generativewith sparsity constraints, often referred to as convolutional dictionary(CDL). Our work is the first to combine ideas from convolutionalmodels and deep learning for data that are naturally modeled with a-Gaussian distribution (e.g., binomial and Poisson). This perspectiveus with a scalable and flexible framework that can be re-purposed forwide range of tasks and assumptions on the generative model. Specifically,iterative optimization procedure for solving CDL, an unsupervised task, isto an unfolded and constrained neural network, with iterativeto the inputs to account for the generative distribution. We alsothat the framework can easily be extended for discriminative training,for a supervised task. We demonstrate 1) that fitting themodel to learn, in an unsupervised fashion, the latent stimulus thatneural spiking data leads to better goodness-of-fit compared to other, 2) competitive performance compared to state-of-the-art algorithmssupervised Poisson image denoising, with significantly fewer parameters,3) gradient dynamics of shallow binomial auto-encoder.