Lab2Pix: Label-Adaptive Generative Adversarial Network for Unsupervised Image Synthesis

Lab2Pix refers to the task of generating photo-realistic images from labels, e.g., semantic labels or sketch labels. Despite inheriting from image-to-image translation, Lab2Pix develops its own characteristics due to the differences between labels and general images. This prevents Lab2Pix task from simply applying general image-to-image translation models. Therefore, we propose an unsupervised framework named Lab2Pix to adaptively synthesize images from labels by elegantly considering the particular properties of label to image synthesis task. Specifically, since the labels contain much less information than the images, we design our generator in a cumulative style which gradually renders synthesized images by fusing features in different levels. Accordingly, the verification process feeds the generated images to a segmentation component and compares the results to the original input label. Furthermore, we propose a sharp enhancement loss, an image consistency loss and a foreground enhancement mask to encourage the network to synthesize photo-realistic images. Experiments conducted on Cityscapes, Facades, Edge2shoes and Edge2handbags datasets demonstrate that our Lab2Pix significantly outperforms existing state-of-the-art unsupervised methods and is even comparable to supervised methods. The source code is available at https://github.com/RoseRollZhu/Lab2Pix.