Tandem: At-Home Behavior Assessment Using Multimodal Signals from the Parent-Child Dyad

The quality of parent-child interactions at an early age has been linked to children's social-emotional development, executive function, and risk for behavior problems. As such, parent-child interactions in naturalistic settings could present a unique opportunity to screen for at-risk behavior in young children, enabling timely and targeted interventions. In this work, we validate the feasibility of using structured at-home play sessions, completed via the Tandem smartphone app, to enable highly accurate and scalable behavioral assessments. We demonstrate that audio and physiological signals recorded during the play session can be used to capture key markers of parent-child interaction dynamics, which are more indicative of at-risk behavior compared to features from each individual alone. We propose novel audio-based dyadic interaction features that significantly outperform conventional speech features at predicting risk for behavior problems, achieving an F1 score of 0.87. Furthermore, we show that dyadic physiological synchrony features, extracted from privacy-preserving wearable sensor data, can classify at-risk behavior with an F1 score of 0.91. Tandem thus sets the stage for automated at-home behavior assessment tools for young children that balance screening accuracy with practical deployment considerations.