CACOLIT: Cross-domain Adaptive Co-learning for Imbalanced Image-to-Image Translation

State-of-the-art unsupervised image-to-image translation (I2I) methods have made great progress on transferring images from a source domain X to a target domain Y. However, training these unsupervised I2I models on imbalanced target domain (e.g., Y with limited samples) usually causes mode collapse, which has not been well solved in current literature. In this work, we propose a new Cross-domain Adaptive Co-learning paradigm, CACOLIT, to alleviate the imbalanced unsupervised I2I training problem. Concretely, CACOLIT first constructs a teacher translation model by introducing an auxiliary domain along with source domain as well as two complementary student translation models formulating an I2I closed loop. Then, the two student models are simultaneously learned by transferring correspondence knowledge from teacher model in an interactive way. With extensive experiments on both human face style transfer and animal face translation tasks, we demonstrate that our adaptive co-learning model effectively transfers correspondence knowledge from teacher model to student models and generates more diverse and realistic images than existing I2I methods both qualitatively and quantitatively.