VIEWS: Entity-Aware News Video Captioning

Existing popular video captioning benchmarks and models often produce generic captions for videos that lack specific identification of individuals, locations, or organizations (namedentities). However, in the case of news videos, the setting is more demanding, requiring the inclusion of such named entities for meaningful summarization. Therefore, we introduce the task of directly summarizing news videos into captions that are entities-aware. To facilitate research in this area, we have collected a large-scale dataset named VIEWS (VIdeo NEWS). Within this task, we face challenges inherent to recognizing named entities and navigating diverse, dynamic contexts, all while relying solely on visual cues. To address these challenges, we propose a model-agnostic approach that enriches visual information extracted from videos with context sourced from external knowledge, enabling the generation of entity-aware captions. We validate the effectiveness of our approach across three video captioning models. Additionally, we conduct a critical analysis of our methodology to gain insights into the complexity of the task, the challenges it presents, and potential avenues for future research. WRAP US Pres arrives at next stop on tour of African nations ADDS presser 1. Wide of helicopter arriving carrying US President Bush 2. Mid of Bush getting out of helicopter with wife, Laura 3. Bush meeting Liberian officials… STORYLINE President George W. Bush said on Thursday that president of the … WRAP US Pres arrives at next stop on tour of African nations ADDS presser 1. Wide of helicopter arriving carrying US President Bush 2. Mid of Bush getting out of helicopter with wife, Laura … US President George W. Bush arrived in Liberia as part of his tour of African nations. He was greeted by Liberian President Ellen Johnson Sirleaf.