Deep Markov Factor Analysis: Towards Concurrent Temporal and Spatial Analysis of fMRI Data

Factor analysis methods have been widely used in neuroimaging to transfer high dimensional imaging data into low dimensional, ideally interpretable representations. However, most of these methods overlook the highly nonlinear and complex temporal dynamics of neural processes when factorizing their imaging data. In this paper, we present deep Markov factor analysis (DMFA), a generative model that employs Markov property in a chain of low dimensional temporal embeddings together with spatial inductive assumptions, all related through neural networks, to capture temporal dynamics in functional magnetic resonance imaging (fMRI) data, and tackle their high spatial dimensionality, respectively. Augmented with a discrete latent, DMFA is able to cluster fMRI data in its low dimensional temporal embedding with regard to subject and cognitive state variability, therefore, enables validation of a variety of fMRI-driven neuroscientific hypotheses. Experimental results on both synthetic and real fMRI data demonstrate the capacity of DMFA in revealing interpretable clusters and capturing nonlinear temporal dependencies in these high dimensional imaging data. Recently, few approaches have been proposed, based on probabilistic generative models, for topographic factorization of fMRI data into a weighted summation of few localized activation sources (i.e., temporal weights and topographic spatial factors), among which topographic factor analysis (TFA) [Manning et al., 2014b] and its multi-subject extension, hierarchical TFA (HTFA) [Manning et al., 2018] , and neural TFA (NTFA) [Sennesh et al., 2020] are the most noted ones. This factorization serves as a necessary preparation step for subsequent statistical analysis that can effectively characterize subject-and stimulus-level variations and reveal task-or cognitive state-related networks in brain. However, TFA approaches assume a prior in which temporal weights are conditionally independent as a function of time, which means they do not encode temporal dynamics. Given the non-linearity and complex time-dependencies inherent in fMRI, a model is required that can capture and represent these dependencies. In this paper, we propose deep Markov factor analysis (DMFA) 1 , a Bayesian model for factorization of fMRI data that learns a deep generative Markovian prior to reason about nonlinear temporal dynamics. This is realized by a chain of low dimensional temporal embeddings related through neural networks. This prior is further augmented by a discrete latent for multimodal dynamical estimation, and clustering subject-and task-level variations directly in its low dimensional temporal embedding. To accommodating high spatial dimensionality, DMFA generatively parameterize spatial factors from a low dimensional spatial latent through neural networks. We evaluate the performance of DMFA on a synthetic and two real large-scale fMRI datasets. Our experiments demonstrate that DMFA uncovers meaningful clusters in these data and achieves better predictive performance for unseen data relative to the state-of-the-art. Related Work Factor Analysis in fMRI: Factor analysis in neuroimaging includes a wide range of approaches for reducing data dimensionality to facilitate their interpretability and computational tractability. Principal component analysis (PCA) [Pearson, 1901] and independent component analysis (ICA) [Comon et al., 1991] are among the most well-known classical factor analysis methods. To accommodate tensor data and mitigate scalability issues, multilinear versions of PCA and ICA have been proposed in Vasilescu and Terzopoulos [2005], Beckmann and Smith [2005], Cichocki [2011], Richard and Montanari [2014], Hopkins et al. [2015]. Specifically, for multi-subject fMRI study, Lee et al. [2008] proposed independent vector analysis (IVA) and Richard et al. [2020] developed MultiView ICA to model shared responses. Likewise, Chen et al. [2015] developed a shared response model (SRM) for aggregating multi-subject fMRI data and highlighting group differences, and Van Kesteren and Kievit [2021] incorporated structured residuals into the exploratory factor analysis (EFA) framework. Karahanoglu and Van De Ville [2015] deconvolved hemodynamic response from rest fMRI time series and then performed temporal clustering on the resulting whole-brain innovation signals to recover the corresponding spatial patterns. However, spatial factors obtained by these methods are unstructured, often have the same size as the images in the original dataset, and may include many small and large voxel clusters across the brain, therefore are not directly interpretable [Manning et al., 2014b]. More important, these methods are permutation invariant along temporal dimension (i.e., do not assume any relationships between temporal dimensions), therefore do not model temporal dynamics [Yu et al., 2016] .