Transportable Representations for Domain Generalization

Generalizing across settings and changing conditions is one of the fundamental problems of AI. One critical assumption in this context is that the testing and training data come from the same distribution, which, despite its popularity in the literature, is often violated in practice. The anchors that allow generalizations to take place are causal and stem from the stability of the mechanisms underlying the system under investigation. Building on the theory of causal transportability introduced by Bareinboim & Pearl, we define the notion of "transportable representations" to provide data structures for allowing a formal analysis of the domain generalization task. We then develop an algorithm to decide whether the distribution of the label, conditioned on the representation, can be computed in terms of the source distributions and assumptions about the commonalities and disparities across source and target domains. Moreover, we relax the assumption of having the graph as the task's input and prove a graphical-invariance duality theorem, delineating the conditions under which certain invariances in the source data can be used as a sound and complete criterion for assessing the generalizability of a classifier. We review the prior literature and show how our findings provide a unifying theoretical perspective over several existing approaches to the domain generalization problem. Considering this background, we note that solving the domain generalization problem can be seen as a two-step process: 1. Evaluation: for a fixed a representation, approximate the distribution of the label conditional on the representation in the target domain, i.e., estimate P * (y | ϕ(X)) or