Concept-Based Automated Grading of CS-1 Programming Assignments

Due to the increasing enrolments in Computer Science programs, teaching of introductory programming needs to be scaled up. This places signi cant strain on teaching resources for programming courses for tasks such as grading of submitted programming assignments. Conventional attempts at automated grading of programming assignment rely on test-based grading which assigns scores based on the number of passing tests in a given test-suite. Since test-based grading may not adequately capture the student's understanding of the programming concepts needed to solve a programming task, we propose the notion of a concept graph which is essentially an abstracted control ow graph. Given the concept graphs extracted from a student's solution and a reference solution, we de ne concept graph matching and comparing of di ering concepts. Our experiments on 1540 student submissions from a publicly available dataset show the e cacy of concept-based grading vis-a-vis test-based grading. Speci cally, the concept based grading is (experimentally) shown to be closer to the grade manually assigned by the tutor. Apart from grading, the concept graph used by our approach is also useful for providing feedback to struggling students, as con rmed by our user study among tutors. CCS CONCEPTS • Applied computing → Computer-assisted instruction; • Software and its engineering → Software testing and debugging.