Duplex Sequence-to-Sequence Learning for Reversible Machine Translation

Sequence-to-sequence learning naturally has two directions. How to effectively utilize supervision signals from both directions? Existing approaches either require two separate models, or a multitask-learned model but with inferior performance. In this paper, we propose REDER (REversible Duplex TransformER), a parameterefficient model and apply it to machine translation. Either end of REDER can simultaneously input and output a distinct language. Thus REDER enables reversible machine translation by simply flipping the input and output ends. Experiments verify that REDER achieves the first success of reversible machine translation, which helps outperform its multitask-trained baselines up to 1.3 BLEU. 1 * Work was done when Zaixiang Zheng was a final-year PhD candidate at Nanjing University and an intern (now FTE) at ByteDance AI Lab; and when Lei Li was also at ByteDance AI Lab. 1 Code is available at https://github.com/zhengzx-nlp/REDER . 35th Conference on Neural Information Processing Systems (NeurIPS 2021).