ECIS-VQG: Generation of Entity-centric Information-seeking Questions from Videos

Previous studies on question generation from videos have mostly focused on generating questions about common objects and attributes and hence are not entity-centric.In this work, we focus on the generation of entity-centric information-seeking questions from videos.Such a system could be useful for videobased learning, recommending "People Also Ask" questions, video-based chatbots, and factchecking.Our work addresses three key challenges: identifying question-worthy information, linking it to entities, and effectively utilizing multimodal signals.Further, to the best of our knowledge, there does not exist a large-scale dataset for this task.Most video question generation datasets are on TV shows, movies, or human activities or lack entitycentric information-seeking questions.Hence, we contribute a diverse dataset of YouTube videos, VIDEOQUESTIONS, consisting of 411 videos with 2265 manually annotated questions.We further propose a model architecture combining Transformers, rich context signals (titles, transcripts, captions, embeddings), and a combination of cross-entropy and contrastive loss function to encourage entity-centric question generation.Our best method yields BLEU, ROUGE, CIDEr, and METEOR scores of 71.3, 78.6, 7.31, and 81.9, respectively, demonstrating practical usability.We make the code and dataset publicly available 1 .