AI meets Mathematics Education: Supporting Instructors in Large Mathematics Classes with Context-Aware AI
Jérémy Valentin Barghorn, Anna Sotnikova, Sacha Friedli, Antoine Bosselut
Abstract
Large-enrollment university courses face persistent challenges in providing timely and scalable instructional support. While generative AI holds promise, its effective use depends on reliability and pedagogical alignment. We present a human-centered case study of AI-assisted support in a Calculus I course, implemented in close collaboration with the course instructor. We developed a system to answer students’ questions on a discussion forum, fine-tuning a lightweight language model on 2,588 historical student–instructor interactions. The model achieved 75.3% accuracy on a benchmark of 150 representative questions annotated by five instructors, and in 36% of cases, its responses were rated equal to or better than instructor answers. Post-deployment student survey (N = 105) indicated that students valued the alignment of the responses with the course materials and their immediate availability, while still relying on the instructor verification for trust. We highlight the importance of hybrid human–AI workflows for safe and effective course support.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on15
- Let's Verify Step by StepHunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards et al.ICLR 2024 · 3,045 citations
- Can Large Language Models Be an Alternative to Human Evaluations?David Cheng-Han Chiang, Hung-yi LeeACL 2023 · 254 citations
- AI-Augmented Brainwriting: Investigating the use of LLMs in group ideationOrit Shaer, Angelora Cooper, Osnat Mokryn, Andrew L. Kun et al.CHI 2024 · 120 citations
- Humans or LLMs as the Judge? A Study on Judgement BiasGuiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang et al.EMNLP 2024 · 37 citations
- Co-designing Large Language Model Tools for Project-Based Learning with K12 EducatorsPrerna Ravi, John Masla, Gisella Kakoti, Grace C. Lin et al.CHI 2025 · 26 citations
Related papers
- Evaluating AI Grading on Real-World Handwritten College Mathematics: A Large-Scale Study Toward a BenchmarkZhiqi Yu, Xingping Liu, Haobin Mao, Mingshuo Liu et al.ICML 2026 · 1 citation
- From Code Generation to Conceptual Learning: Student Use of LLMs in a Web Programming CourseHajara-Yasmin Isa, Matthew Weston, Muhammad Rizky Wellyanto, Ishita Karna et al.CHI 2026 · 1 citation
- From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement LearningDavid Dinucu-Jianu, Jakub Macina, Nico Daheim, Ido Hakimi et al.EMNLP 2025
- Toward Scalable and Responsible Integration of Course-Specific AI Tutors: Instructor Experiences with a Campus-Wide PlatformEunhye Grace Ko, Hakeoung Hannah Lee, Anjali Singh, Lily Boddy et al.CHI 2026 · 2 citations
- "Here the GPT made a choice, and every choice can be biased": How Students Critically Engage with LLMs through End-User Auditing ActivitySnehal Prabhudesai, Ananya Prashant Kasi, Anmol Mansingh, Anindya Das Antar et al.CHI 2025 · 15 citations
