Mapping conditional distributions for domain adaptation under generalized target shift

We consider the problem of unsupervised domain adaptation (UDA) between a source and a target domain under conditional and label shift a.k.a Generalized Target Shift (GeTarS). Unlike simpler UDA settings, few works have addressed this challenging problem. Recent approaches learn domain-invariant representations, yet they have practical limitations and rely on strong assumptions that may not hold in practice. In this paper, we explore a novel and general approach to align pretrained representations, which circumvents existing drawbacks. Instead of constraining representation invariance, it learns an optimal transport map, implemented as a NN, which maps source representations onto target ones. Our approach is flexible and scalable, it preserves the problem's structure and it has strong theoretical guarantees under mild assumptions. In particular, our solution is unique, matches conditional distributions across domains, recovers target proportions and explicitly controls the target generalization risk. Through an exhaustive comparison on several datasets, we challenge the state-of-the-art in GeTarS. INTRODUCTION Unsupervised Domain Adaptation (UDA) methods (Pan & Yang, 2010) train a classifier with labelled samples from a source domain S such that its risk on an unlabelled target domain T is low. This problem is ill-posed and simplifying assumptions were considered. Initial contributions focused on three settings which decompose differently the joint distribution over input and label ), target shift (TarS) (Zhang et al., 2013) with p S (Y ) = p T (Y ), p S (X|Y ) = p T (X|Y ) and conditional shift (Zhang et al., 2013) with p S (X|Y ) = p T (X|Y ), p S (Y ) = p T (Y ). These assumptions are restrictive for real-world applications and were extended into model shift when p S (Y |X) = p T (Y |X), p S (X) = p T (X) (Wang & Schneider, 2014; 2015) and generalized target shift (GeTarS) (Zhang et al., 2013) when p S (X|Y ) = p T (X|Y ), p S (Y ) = p T (Y ). We consider GeTarS where a key challenge is to map the source domain onto the target one to minimize both conditional and label shifts, without using target labels. The current SOTA in Gong et al. (2016); Combes et al. (2020); Rakotomamonjy et al. (2021); Shui et al. ( 2021 ) learns domain-invariant representations and uses estimated class-ratios between domains as importance weights in the training loss. However, this approach has several limitations. First, it updates representations through adversarial alignment which is prone to well-known instabilities, especially on applications where there is no established Deep Learning architectures e.g. click-through-rate prediction, spam filtering etc. in contrast to vision. Second, to transfer representations, the domain-invariance constraint breaks the original problem structure and it was shown that this may degrade the discriminativity of target representations (Liu et al., 2019) . Existing approaches that consider this issue (Xiao et al., 2019; Li et al., 2020; Chen et al., 2019) were not applied to GeTarS. Finally, generalization guarantees are derived under strong assumptions, detailed in Section 2.3, which may not hold in practice. In this paper, we address these limitations with a new general approach, named Optimal Sample Transformation and Reweight (OSTAR), which maps pretrained representations using Optimal Transport (OT). OSTAR proposes an alternative to constraining representation invariance and performs jointly three operations: given a pretrained encoder, (i) it learns an OT map, implemented as a neural network (NN), between encoded source and target conditionals, (ii) it estimates target proportions for sample reweighting and (iii) it learns a classifier for the target domain using source labels.