Open-ended Human Activity Understanding via LLM-assisted Motion Decomposition and Semantic Fusion

Human activity is a fundamental component of intelligent interactive systems. Existing human activity recognition (HAR) follows a closed-ended assumption—defining a fixed set of activity classes and classifying each sensor signal into one of them—yet real-world human behavior is complex and ever-changing and thus cannot be fully represented by any fixed set. We present a new paradigm for open-ended human activity understanding (HAU), shifting the goal from closed-set classification to understanding activities via their meta-motions. To achieve this, we design MOSAIC, a motion decomposition and semantic fusion framework that transforms raw sensor streams into structured meta-motion representations and subsequently generates fine-grained natural-language descriptions. Additionally, we propose an LLM-driven training strategy for sensor-language alignment, enabling effective cross-modal supervision in the absence of real-world training data. Finally, we leverage LLM-assisted reasoning to recognize complex high-level activities based on the understanding of underlying motions and their composition. Our model demonstrates strong generalization and performance across 18 public HAR datasets, outperforming the best baseline by up to 19.6% in unseen scenarios.