Learning Causal Semantic Representation for Out-of-Distribution Prediction

Conventional supervised learning methods, especially deep ones, are found to be sensitive to out-of-distribution (OOD) examples, largely because the learned representation mixes the semantic factor with the variation factor due to their domain-specific correlation, while only the semantic factor causes the output. To address the problem, we propose a Causal Semantic Generative model (CSG) based on a causal thought where the two factors are modeled separately, and develop methods to learn it on a single training domain and predict in a test domain without (OOD generalization) or with unsupervised data (domain adaptation). We prove that under proper conditions CSG identifies the semantic factor by fitting training data, and this semantic identification guarantees the boundedness of OOD generalization error and the success of adaptation. The methods and theory are based on the invariance principle of causal generative mechanisms, which is more fundamental and general than inference invariance. The methods come from a novel design for both efficient learning and easy prediction, following the first principle of variational Bayes and the graphical structure of CSG. Empirical study demonstrates the improved test accuracy for OOD generalization and domain adaptation.