ACFlow: Flow Models for Arbitrary Conditional Likelihoods

Understanding the dependencies among features of a dataset is at the core of most unsupervised learning tasks. However, a majority of generative modeling approaches are focused solely on the joint distribution p(x) and utilize models where it is intractable to obtain the conditional distribution of some arbitrary subset of features x u given the rest of the observed covariates x o : p(x u | x o ). Traditional conditional approaches provide a model for a fixed set of covariates conditioned on another fixed set of observed covariates. Instead, in this work we develop a model that is capable of yielding all conditional distributions p(x u | x o ) (for arbitrary x u ) via tractable conditional likelihoods. We propose a novel extension of (change of variables based) flow generative models, arbitrary conditioning flow models (ACFlow). ACFlow can be conditioned on arbitrary subsets of observed covariates, which was previously infeasible. We further extend ACFlow to model the joint distributions p(x) and arbitrary marginal distributions p(x u ). We also apply ACFlow to the imputation of features, and develop a unified platform for both multiple and single imputation by introducing an auxiliary objective that provides a principled single "best guess" for flow models. Extensive empirical evaluations show that our model achieves state-of-the-art performance in modeling arbitrary conditional distributions in addition to both single and multiple imputation in synthetic and real-world datasets.