CrowdAct: Achieving High-Quality Crowdsourced Datasets in Mobile Activity Recognition

In this study, we propose novel gamified active learning and inaccuracy detection for crowdsourced data labeling for an activity recognition system using mobile sensing (CrowdAct). First, we exploit active learning to address the lack of accurate information. Second, we present the integration of gamification into active learning to overcome the lack of motivation and sustained engagement. Finally, we introduce an inaccuracy detection algorithm to minimize inaccurate data. To demonstrate the capability and feasibility of the proposed model in realistic settings, we developed and deployed the CrowdAct system to a crowdsourcing platform. For our experimental setup, we recruited 120 diverse workers. Additionally, we gathered 6,549 activity labels from 19 activity classes by using smartphone sensors and user engagement information. We empirically evaluated the quality of CrowdAct by comparing it with a baseline using techniques such as machine learning and descriptive and inferential statistics. Our results indicate that CrowdAct was effective in improving activity accuracy recognition, increasing worker engagement, and reducing inaccurate data in crowdsourced data labeling. Based on our findings, we highlight critical and promising future research directions regarding the design of efficient activity data collection with crowdsourcing.