Manifold Adversarial Augmentation for Neural Machine Translation

Improving the robustness of neural machine translation models on variations of input sentences is an active area of research. In this paper, we propose a simple data augmentation approach by sampling virtual sentences from the vicinity distributions in higher-level representations, constructed either from individual training samples via adversarial learning or pairs of training samples through mixup. By simplifying and extending previous work that operates at the token level, our method can construct virtual training samples in a broader space and achieve improved translation accuracy compared to the previous stateof-the-art. In addition, we present a simple variation of the mixup strategy to better utilize the pseudo training samples created from backtranslation, obtaining further improvement in performance.