Beyond Item Selection: Rethinking Ability Evolution in Computerized Adaptive Testing

Computerized Adaptive Testing (CAT), as a core assessment technique in intelligent education, aims to enhance the efficiency and precision of learner evaluation by adaptively selecting questions that best align with the examinee's current knowledge state. Indeed, recent years have witnessed substantial progress in the design and refinement of the fundamental cognitive diagnosis models and the question selection strategies, leading to promising results. However, the evolutionary dynamics of examinee abilities and the underlying updating mechanisms throughout the adaptive testing process have received limited attention. To this end, this paper investigates the impact of update mechanisms with varying granularities on the accuracy of ability estimation, with a particular focus on step-based and batch-based strategies in the context of assessment diagnosis. Specifically, we begin by conducting an approximation analysis to compare the changes in ability estimates induced by the two strategies, followed by a theoretical analysis of their update trajectories through the lens of error bounds. To trade-off the adaptability and stability during the assessment process, we propose SUAT, a Smooth-step Updating strategy for more effective Adaptive Testing. Finally, we conduct extensive experiments on two real-world educational datasets of varying scales, and provide a detailed analysis and discussion of the differences among the different update mechanisms, and their potential applications.