Structure-Aware Pre-Training for Table-to-Text Generation

Table-to-text generation is a subtask of datato-text generation which aims to generate naltural language text based on input table. Pretraining techniques have achieved great success on table-to-text generation. However, the pre-trained models used in previous works are typically trained on free-form natural language text while the input of table-to-text task is structured table. In this paper, we propose STTP, a pre-trained model that is trained with tables and their contexts. The STTP model can understand the structured input table and generate fluent text. Experiments on two datasets show the efficacy of our model.