A Simple and Effective Unified Encoder for Document-Level Machine Translation

Most of the existing models for documentlevel machine translation adopt dual-encoder structures. The representation of the source sentences and the document-level contexts 1 are modeled with two separate encoders. Although these models can make use of the document-level contexts, they do not fully model the interaction between the contexts and the source sentences, and can not directly adapt to the recent pre-training models (e.g., BERT) which encodes multiple sentences with a single encoder. In this work, we propose a simple and effective unified encoder that can outperform the baseline models of dualencoder models in terms of BLEU and ME-TEOR scores. Moreover, the pre-training models can further boost the performance of our proposed model.