Score Matching with Missing Data

Score matching is a vital tool for learning the distribution of data with applications across many areas including diffusion processes, energy based modelling, and graphical model estimation. Despite all these applications, little work explores its use when data is incomplete. We address this by adapting score matching (and its major extensions) to work with missing data in a flexible setting where data can be partially missing over any subset of the coordinates. We provide two separate score matching variations for general use, an importance weighting (IW) approach, and a variational approach. We provide finite sample bounds for our IW approach in finite domain settings and show it to have especially strong performance in small sample lower dimensional cases. Complementing this, we show our variational approach to be strongest in more complex highdimensional settings which we demonstrate on graphical model estimation tasks on both real and simulated data. X p(x)dx -λ and p(x λ |x -λ ) being the conditional density of X λ |X -λ = x -λ for example. Now that we have introduced our notation we can move onto the key area of focus for our work, score matching. Score Matching First proposed by (Hyvärinen, 2005) , score matching aims to learn the gradient of the log-density (score). The advantage of this framework over full density approaches such as maximum likelihood estimation (MLE) is that we are not restricted to parametric models which integrate to 1. This allows us to be much more flexible in how we parameterise in turn making high dimensional distribution modelling more feasible. We now introduce the approach. Let X be a RV over R d with PDF p. We say that q is the unnormalised density of X if N -1 • q(x) = p(x) where