Understanding Behind the Smile of Emotion Workers: Detecting After-Call Stress in Call Agents

Call agents, a representative group of emotion workers, must manage emotions under constrained autonomy, yet workplace stress sensing has primarily centered on knowledge work. We ask how the task‑aligned cycle of emotional labor, alternating customer interaction (CI) and non‑customer interaction (nCI), shapes stress and how it manifests in data. We conducted a month-long in-the-wild formative mixed-methods study with professional call agents, collecting structured task logs, environmental and behavioral signals, and per-call stress self-reports, followed by semi-structured interviews. Task logs, used as a new sensor modality, were incorporated as primary sensing signals, and task-related features were extracted by respecting CI boundaries for modeling. Our results showed that a short 5-minute windowing approach was comparable to task-aligned windowing using multimodal sensors, with task-related features being considered the most important across all generalized models. Personalized models improved further and shifted importance toward diverse data sources, revealing individual differences in preparation patterns. Interviews support those findings, reveal key modelling challenges, and highlight potential benefits of semi-automated self-tracking. We discuss implications for timing interventions at breakpoints suited for work patterns, and ethically deploying stress support for emotion workers.