Lune

ICLR2026Top-tier venue

A Bayesian Nonparametric Framework For Learning Disentangled Representations

Vaishnavi Patil, Siddhi Patil, Matthew Evanusa, Amit Kumar Kundu, Cornelia Fermüller, Joseph F. JáJá

Abstract

Disentangled representation learning aims to identify and organize the underlying sources of variation in observed data. However, learning disentangled representations from observational data alone without any additional supervision necessitates inductive biases to solve the fundamental identifiability problem of uniquely recovering the true latent structure and parameters of the data-generating process. Existing methods address this by imposing heuristic inductive biases that typically lack these theoretical identifiability guarantees. Additionally, these methods rely on strong regularization to impose these inductive biases, creating an inherent trade-off in which stronger regularization improves disentanglement but limits the latent capacity to represent underlying variations. To address both challenges, we propose a principled generative model with a Bayesian nonparametric hierarchical mixture prior that embeds inductive biases within a provably identifiable framework for unsupervised disentanglement. Specifically, the hierarchical mixture prior imposes the structural constraints necessary for identifiability guarantees, while the nonparametric formulation allows the latent representation to scale with infinite capacity to faithfully represent the complete set of underlying variations without violating these structural constraints. To enable tractable inference under this nonparametric hierarchical prior, we develop a structured variational inference framework with a nested variational family that both preserves the hierarchical structure of the identifiable generative model and approximates the expressiveness of the nonparametric prior. We evaluate our proposed probabilistic model on standard disentanglement benchmarks, 3DShapes and MPI3D datasets characterized by diverse source variation distributions, to demonstrate that our method consistently outperforms strong baseline models through structural biases and a unified objective function, obviating the need for auxiliary regularization constraints or careful hyperparameter tuning.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding cites Weakly Supervised Disentanglement with Guarantees
  • Identifiability of deep generative models without auxiliary information cites Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
  • Identifiability of deep generative models without auxiliary information cites Posterior Collapse and Latent Variable Non-identifiability
  • Identifiability of deep generative models without auxiliary information cites Weakly-Supervised Disentanglement Without Compromises
  • Weakly-Supervised Disentanglement Without Compromises cites Weakly Supervised Disentanglement with Guarantees
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on13

  1. Weakly-Supervised Disentanglement Without CompromisesFrancesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf et al.ICML 2020 · 361 citations
  2. Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse CodingDavid A. Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov et al.ICLR 2021 · 156 citations
  3. Weakly Supervised Disentanglement with GuaranteesRui Shu, Yining Chen, Abhishek Kumar, Stefano Ermon et al.ICLR 2020 · 148 citations
  4. Posterior Collapse and Latent Variable Non-identifiabilityYixin Wang, David M. Blei, John P. CunninghamNeurIPS 2021 · 97 citations
  5. Identifiability of deep generative models without auxiliary informationBohdan Kivva, Goutham Rajendran, Pradeep Ravikumar, Bryon AragamNeurIPS 2022 · 87 citations

Related papers

  1. An Identifiable Double VAE For Disentangled RepresentationsGraziano Mita, Maurizio Filippone, Pietro MichiardiICML 2021 · 39 citations
  2. Bayes-Factor-VAE: Hierarchical Bayesian Deep Auto-Encoder Models for Factor DisentanglementMinyoung Kim, Yuting Wang, Pritish Sahu, Vladimir PavlovicICCV 2019 · 29 citations
  3. VDSM: Unsupervised Video Disentanglement With State-Space Modeling and Deep Mixtures of ExpertsMatthew J. Vowels, Necati Cihan Camgöz, Richard BowdenCVPR 2021
  4. Continual Unsupervised Disentangling of Self-Organizing RepresentationsZhiyuan Li, Xiajun Jiang, Ryan Missel, Prashnna Kumar Gyawali et al.ICLR 2023
  5. Geometric Inductive Biases for Identifiable Unsupervised Learning of Disentangled RepresentationsZiqi Pan, Li Niu, Liqing ZhangAAAI 2023 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines