EduVidQA: Generating and Evaluating Long-form Answers to Student Questions based on Lecture Videos

As digital platforms redefine educational paradigms, ensuring interactivity remains vital for effective learning. This paper explores using Multimodal Large Language Models (MLLMs) to automatically respond to student questions from online lectures -a novel question answering task of real world significance. We introduce the EduVidQA Dataset with 5252 question-answer pairs (both synthetic and realworld) from 296 computer science videos covering diverse topics and difficulty levels. To understand the needs of the dataset and task evaluation, we empirically study the qualitative preferences of students, which we provide as an important contribution to this line of work. Our benchmarking experiments consist of 6 stateof-the-art MLLMs, through which we study the effectiveness of our synthetic data for finetuning, as well as showing the challenging nature of the task. We evaluate the models using both text-based and qualitative metrics, thus showing a nuanced perspective of the models' performance, which is paramount to future work. This work not only sets a benchmark for this important problem, but also opens exciting avenues for future research in the field of Natural Language Processing for Education. * indicates equal supervision Code and data: https://github.com/sourjyadip/ eduvidqa-emnlp25 0 5 10 15 20 25 30 2 Related Work 2.1 Educational Video QA Datasets Dataset Video Type Answer Type Avg Vid Length Reasoning Type TutorialVQA Tutorial Open Ended 1488 secs Comprehension How2QA Tutorial MCQ 17.45 secs Comprehension HowToVQA Tutorial MCQ 12.1 secs Comprehension YTCommentQA Tutorial Open Ended 524 secs Knowledge EduVidQA Lecture Open Ended 4054 secs Evaluation https://pypi.org/project/python-youtube/