The Hidden Workload: Student Data Work in Multimodal Algorithmic Evaluations

As algorithmic systems increasingly mediate human activities across diverse domains, they shift more responsibility for data collection onto users, fundamentally altering the nature of data work. This paper examines the implications of this shift by investigating student-led data collection and automated feedback interpretation using a mobile, multimodal learning analytics (MMLA) tool designed to coach oral presentation skills. Our findings reveal that while this user-controlled data collection provides greater flexibility, it also imposes speculative labor, compelling students to adjust their behavior to align with perceived standards of good data even when such changes are unwarranted. The study highlights the often-overlooked informal labor involved in managing the socio-material conditions of data collection, emphasizing the need for MMLA tools that offer adaptive support and guidance. These insights extend to algorithmic system design in educational and professional contexts, advocating for systems that balance user autonomy with workload-minimizing guidance to achieve equitable accountability.