IoTMosaic: Inferring User Activities from IoT Network Traffic in Smart Homes

Recent advances in cyber-physical systems, artificial intelligence, and cloud computing have driven the wide deployment of Internet-of-things (IoT) in smart homes. As IoT devices often directly interact with the users and environments, this paper studies if and how we could explore the collective insights from multiple heterogeneous IoT devices to infer user activities for home safety monitoring and assisted living. Specifically, we develop a new system, namely IoTMosaic, to first profile diverse user activities with distinct IoT device event sequences, which are extracted from smart home network traffic based on their TCP/IP data packet signatures. Given the challenges of missing and out-of-order IoT device events due to device malfunctions or varying network and system latencies, IoTMosaic further develops simple yet effective approximate matching algorithms to identify user activities from real-world IoT network traffic. Our experimental results on thousands of user activities in the smart home environment over two months show that our proposed algorithms can infer different user activities from IoT network traffic in smart homes with the overall accuracy, precision, and recall of 0.99, 0.99, and 1.00, respectively.

Recent advances in cyber-physical systems, artificial intelligence, and cloud computing have driven the rapid growth and deployment of IoT devices in smart homes. Although there is a rich literature in studying traffic patterns [17,34], security and privacy challenges [8,9,11,14,15,20,39], and device events and functions [6,7,10,24,31,33,38] of individual IoT devices, little effort has been devoted to exploring IoT network traffic for inferring user activities. The accurate knowledge and awareness of user activities in smart homes is crucial for home safety monitoring and assisted living, e.g., a motion sensor's motion detection event at the front door followed by a smart lock's open event indicating a person entering the home.

Towards filling this research gap, this paper introduces a new system, namely IoTMosaic, for inferring user activities from IoT network traffic in the smart home environment. IoTMosaic first recognizes and generates the signatures of user activities by characterizing each user activity with an ordered sequence of IoT device events via controlled experiments in smart homes. Based on the signatures of user activities, we could search and match them from IoT device event streams extracted from IoT network traffic for user activity inference. However, the sequence of IoT devices events for a given