Negative Sampling in Semi-Supervised learning

We introduce Negative Sampling in Semi-Supervised Learning (NS 3 L), a simple, fast, easy to tune algorithm for semi-supervised learning (SSL). NS 3 L is motivated by the success of negative sampling/contrastive estimation. We demonstrate that adding the NS 3 L loss to state-of-theart SSL algorithms, such as the Virtual Adversarial Training (VAT), significantly improves upon vanilla VAT and its variant, VAT with Entropy Minimization. By adding the NS 3 L loss to Mix-Match, the current state-of-the-art approach on semi-supervised tasks, we observe significant improvements over vanilla MixMatch. We conduct extensive experiments on the CIFAR10, CI-FAR100, SVHN and STL10 benchmark datasets. Finally, we perform an ablation study for NS 3 L regarding its hyperparameter tuning.