SummScreen: A Dataset for Abstractive Screenplay Summarization

We introduce SUMMSCREEN, a summarization dataset comprised of pairs of TV series transcripts and human written recaps. The dataset provides a challenging testbed for abstractive summarization for several reasons. Plot details are often expressed indirectly in character dialogues and may be scattered across the entirety of the transcript. These details must be found and integrated to form the succinct plot descriptions in the recaps. Also, TV scripts contain content that does not directly pertain to the central plot but rather serves to develop characters or provide comic relief. This information is rarely contained in recaps. Since characters are fundamental to TV series, we also propose two entity-centric evaluation metrics. Empirically, we characterize the dataset by evaluating several methods, including neural models and those based on nearest neighbors. An oracle extractive approach outperforms all benchmarked models according to automatic metrics, showing that the neural models are unable to fully exploit the input transcripts. Human evaluation and qualitative analysis reveal that our nonoracle models are competitive with their oracle counterparts in terms of generating faithful plot events and can benefit from better content selectors. Both oracle and non-oracle models generate unfaithful facts, suggesting future research directions. 1 * Work done while the author was at the University of Chicago 1 Code, data, and trained models are available at https: //github.com/mingdachen/SummScreen [ The apartment ] Sheldon : What color would you like to be ? Leonard : Well , I 'd like to be green , but you know you always take it . Sheldon : That 's not true . Any color 's fine with me . Yeah , I could be aa combination of blue and yellow . Leonard : Blue and yellow make green . Sheldon : Well , then it 's settled . Penny : Hi . Ready to go ? Sheldon : Oh , good news , we ordered lunch , so we can all stay here and play Lord of the Rings Risk . Amy : Sheldon , we said that we would play games with you tonight . Sheldon : Oh , no , we 'll still be playing it tonight , this game can easily take eight hours . Penny : Sweetie , you really thought I 'd want to do this ? Leonard : No . Penny : Well , did you tell him that ? Leonard : Yes . Penny : Did you say it out loud with words ? Leonard : No . Penny : I do n't want to spend the whole day playing a board game . … Sheldon and Leonard are happy playing a board game until Amy and Penny say they are tired of doing what the guys want …