Learning Behaviors Mediate the Effect of AI-powered Support for Metacognitive Calibration on Learning Outcomes

Students struggle with accurately assessing their own performance, especially given little training to do so. We propose an AI-powered training tool to help students improve "metacognitive calibration, " or the ability to accurately predict their own learning, potentially enhancing learning outcomes by enabling students' use of metacognitioninformed learning behaviors. We present results from a randomized controlled trial (N = 133) assessing the effectiveness of the tool in a college-level computer-based learning environment. The AIdriven tool significantly improved learning gains compared to the control group by 8.9% (t = -2.384, p = .019), and this effect was significantly mediated by learning behaviors. Overconfident students who received the intervention showed significantly greater metacognitive calibration improvement than the control group by 4.1% (t = 2.001, p = .049). These insights highlight the value of AIpowered metacognitive calibration training and the importance of promoting specific metacognition-informed learning behaviors in computer-based learning.