EchoScriptor: Automatic Lifelogging Narratives via Activity-Based Audio-Language Model

Automatic, camera-free lifelogging offers new opportunities for memory rehabilitation, personal informatics, and assistive technologies. However, most existing approaches limit daily activities to isolated event labels, offering little context and lacking the narrative coherence essential for effective lifelogging. Recent advances in audio–language models combine foundation audio processing with language-based reasoning, enabling open-ended sound understanding. We introduce EchoScriptor, an end-to-end system that transforms raw in-home audio into context-aware natural-language descriptions, generating coherent narrative lifelogs of activities and acoustic contexts. In moment-level evaluation, EchoScriptor achieved 94.15% activity recognition and 89.25% background recognition accuracy, and at the summary level, achieved an F1 score of 0.92, outperforming the classifier+LLM baseline. In our user study with 20 participants across 10 household activity videos, EchoScriptor summaries were consistently rated highly, approaching the perceived utility of human-written ones. By advancing from event detection to narrative understanding, EchoScriptor establishes a significant step toward automated, unobtrusive, context-aware lifelogging technologies.