SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches
Ehsan Latif, Zirak Khan, Xiaoming Zhai
摘要
Scientific sketches (e.g., models) offer a powerful lens into students' conceptual understanding, yet AI-powered automated assessment of such free-form, visually diverse artifacts remains a critical challenge. Existing solutions often treat sketch evaluation as either an image classification task or monolithic vision-language models, which lack interpretability, pedagogical alignment, and adaptability across cognitive levels. To address these limitations, we present SKETCHMIND, a cognitively grounded, multi-agent framework for evaluating and improving studentdrawn scientific sketches. SKETCHMIND introduces Sketch Reasoning Graphs (SRGs), semantic graph representations that embed domain concepts and Bloom's taxonomy-based cognitive labels. The system comprises modular agents responsible for rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. We evaluate SKETCHMIND on a curated dataset of 3,575 student-generated sketches across six science assessment items with different highest order of Bloom's level that require students to draw models to explain phenomena. Compared to baseline GPT-4o performance without SRG (average accuracy: 55.6%), and with bSRG integration achieves 77.1% average accuracy (+21.4% average absolute gain). We also demonstrate that multi-agent orchestration with SRG enhances SKETCHMIND performance, for example, a SketchMind with GPT-4.1 gains an average 8.9% increase in sketch prediction accuracy, outperforming single-agent pipelines across all items. Human evaluators rated the feedback and co-created sketches generated by SKETCHMIND with GPT-4.1, which achieved an average of 4.1 out of 5, significantly higher than those of baseline models (e.g., 2.3 for GPT-4o). Experts noted the system's potential to meaningfully support conceptual growth through guided revision. Our code and (pending approval) dataset will be released to support reproducibility and future research in AI-driven education.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper5
- SKED: Sketch-guided Text-based 3D EditingAryan Mikaeili, Or Perel, Mehdi Safaee, Daniel Cohen-Or 等ICCV 2023 · 被引用 83 次
- Sketch2Saliency: Learning to Detect Salient Objects from Human DrawingsAyan Kumar Bhunia, Subhadeep Koley, Amandeep Kumar, Aneeshan Sain 等CVPR 2023
- ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large Language ModelsHao Yin, Guangzong Si, Zilei WangCVPR 2025
- Data-Free Sketch-Based Image RetrievalAbhra Chaudhuri, Ayan Kumar Bhunia, Yi-Zhe Song, Anjan DuttaCVPR 2023
- Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language ModelsJiacong Xu, Shao-Yuan Lo, Bardia Safaei, Vishal M. Patel 等CVPR 2025
相关 Paper
- SketchGPT: A Sketch-based Multimodal Interface for Application-Agnostic LLM InteractionZeyuan Huang, Cangjun Gao, Yaxian Shan, Haoxiang Hu 等UIST 2025 · 被引用 8 次
- EvoSci: A Bio-Inspired Multi-Agent Framework for the Evolution of Scientific DiscoveryXiaoyu Xiong, Yuqi Ren, Deyi XiongACL 2026 · 被引用 1 次
- CSG: Cognitive Structure Generation for Intelligent EducationHengnian Gu, Zhifu Chen, Yuxin Chen, Jin Zhou 等ICML 2026
- GIVE: Structured Reasoning of Large Language Models with Knowledge Graph Inspired Veracity ExtrapolationJiashu He, Mingyu Derek Ma, Jinxuan Fan, Dan Roth 等ICML 2025
- Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language ModelsYushi Hu, Weijia Shi, Xingyu Fu, Dan Roth 等NeurIPS 2024 · 被引用 373 次
