Multilayer Matrix Factorization via Dimension-Reducing Diffusion Variational Inference
Abstract
Multilayer matrix factorization (MMF) has recently emerged as a generalized model of, and potentially a more expressive approach than, the classic matrix factorization. This paper considers MMF under a probabilistic formulation, and our focus is on inference methods under variational inference. The challenge in this context lies in determining a variational process that leads to a computationally efficient and accurate approximation of the maximum likelihood inference. One well-known example is the variational autoencoder (VAE), which uses neural networks for the variational process. In this work, we take insight from variational diffusion models in the context of generative models to develop variational inference for MMF. We propose a dimension-reducing diffusion process that results in a new way to interact with the layered structures of the MMF model. Experimental results demonstrate that the proposed diffusion variational inference method leads to improved performance scores compared to several existing methods, including the VAE.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- High-Resolution Image Synthesis with Latent Diffusion Models cites NVAE: A Deep Hierarchical Variational Autoencoder
- High-Resolution Image Synthesis with Latent Diffusion Models cites Denoising Diffusion Probabilistic Models
- High-Resolution Image Synthesis with Latent Diffusion Models cites Score-Based Generative Modeling through Stochastic Differential Equations
- Diffusion Recommender Model cites High-Resolution Image Synthesis with Latent Diffusion Models
- Diffusion Recommender Model cites Denoising Diffusion Probabilistic Models
- Score-Based Generative Modeling through Stochastic Differential Equations cites Denoising Diffusion Probabilistic Models
Builds on8
- Denoising Diffusion Probabilistic ModelsJonathan Ho, Ajay Jain, Pieter AbbeelNeurIPS 2020 · 35,902 citations
- High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser et al.CVPR 2022 · 13,123 citations
- Score-Based Generative Modeling through Stochastic Differential EquationsYang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar et al.ICLR 2021 · 1,270 citations
- NVAE: A Deep Hierarchical Variational AutoencoderArash Vahdat, Jan KautzNeurIPS 2020 · 1,141 citations
- Diffusion Recommender ModelWenjie Wang, Yiyan Xu, Fuli Feng, Xinyu Lin et al.SIGIR 2023 · 281 citations
Related papers
- DiffEnc: Variational Diffusion with a Learned EncoderBeatrix Miranda Ginn Nielsen, Anders Christensen, Andrea Dittadi, Ole WintherICLR 2024 · 16 citations
- DyDiff-VAE: A Dynamic Variational Framework for Information Diffusion PredictionRuijie Wang, Zijie Huang, Shengzhong Liu, Huajie Shao et al.SIGIR 2021 · 41 citations
- Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations ModelingTianyu Xie, Shuchen Xue, Zijin Feng, Tianyang Hu et al.ICLR 2026 · 14 citations
- Variational Information Diffusion for Probabilistic Cascades PredictionFan Zhou, Xovee Xu, Kunpeng Zhang, Goce Trajcevski et al.INFOCOM 2020 · 41 citations
- Deep Markov Factor Analysis: Towards Concurrent Temporal and Spatial Analysis of fMRI DataAmirreza Farnoosh, Sarah OstadabbasNeurIPS 2021 · 9 citations
