Retrieval Enhanced Model for Commonsense Generation

Commonsense generation is a challenging task of generating a plausible sentence describing an everyday scenario using provided concepts. Its requirement of reasoning over commonsense knowledge and compositional generalization ability even puzzles strong pre-trained language generation models. We propose a novel framework using retrieval methods to enhance both the pre-training and fine-tuning for commonsense generation. We retrieve prototype sentence candidates by concept matching and use them as auxiliary input. For finetuning, we further boost its performance with a trainable sentence retriever. We demonstrate experimentally on the large-scale Common-Gen benchmark that our approach achieves new state-of-the-art results. 1 * Work done during internship at Microsoft. 1 The code and data are available at https://github. com/HanNight/RE-T5 Concept Set #1: dog, frisbee, catch, throw Gold Target Sentences: A dog leaps to catch a thrown frisbee. The dog catches the frisbee when the boy throws it. A man throws away his dog 's favorite frisbee expecting him to catch it in the air. Concept Set #2: lake, shore, canoe Gold Target Sentences: Canoe on a shore of lake. Canoe on shore with rainbow across the lake. Several canoes parked in the grass on the shore of a lake.