KATN: Key Activity Detection via Inexact Supervised Learning

Leveraging sensor data of mobile devices and wearables, activity detection is a critical task in various intelligent systems. Most recent work train deep models to improve the accuracy of recognizing specific human activities, which, however, rely on specially collected and accurately labeled sensor data. It is labor-intensive and time-consuming to collect and label large-scale sensor data that cover various people, mobile devices, and environments. In production scenarios, on the one hand, the lack of accurately labeled sensor data poses significant challenges to the detection of key activities; on the other hand, massive continuously generated sensor data attached with inexact information is severely underutilized. For example, in an on-demand food delivery system, detecting the key activity that the rider gets off his/her motorcycle to hand food over to the customer is essential for predicting the exact delivery time. Nevertheless, the system has only the raw sensor data and the clicking "finish delivery" events, which are highly relevant to the key activity but very inexact, since different riders may click "finish delivery" at any time in the last-mile delivery. Without exact labels of key activities, in this work, we propose a system, named KATN, to detect the exact regions of key activities based on inexact supervised learning. We design a novel siamese key activity attention network (SAN) to learn both discriminative and detailed sequential features of the key activity under the supervision of inexact labels. By interpreting the behaviors of SAN, an exact time estimation method is devised. We also provide a personal adaptation mechanism to cope with diverse habits of users. Extensive experiments on both public datasets and data from a real-world food delivery system testify the significant advantages of our design. Furthermore, based on KATN, we propose a novel user-friendly annotation mechanism to facilitate the annotation of large-scale sensor data for a wide range of applications.