SleepMaMi: A Universal Sleep Foundation Model for Integrating Macro- and Micro-structures
Keondo Park, Younghoon Na, Yourim Choi, Hyunwoo Ryu, Hyun-Woo Shin, Hyung-Sin Kim
摘要
While the shift toward unified foundation models has revolutionized many deep learning domains, sleep medicine remains largely restricted to taskspecific models that focus on localized microstructure features. These approaches often neglect the rich, multi-modal context of Polysomnography (PSG) and fail to capture the global macrostructure of a full night's sleep. To address this, we introduce SleepMaMi, a Sleep Foundation Model engineered to master both hour-long sleep architectures and fine-grained signal morphologies. Our framework utilizes a hierarchical dual-encoder design: a Macro-Encoder to model full-night temporal dependencies and a Micro-Encoder to capture short-term characteristics from biosignals. Macro-Encoder is trained via Demographic-Guided Contrastive Learning, which aligns overnight sleep patterns with objective subject metadata, such as age, sex, and BMI to refine global representations. Micro-Encoder is optimized via a hybrid Masked Autoencoder (MAE) and multi-modal contrastive objective. Pre-trained on a massive corpus of >20,000 PSG recordings (158K hours), Sleep-MaMi outperforms or matches state-of-the-art foundation models across a diverse suite of downstream tasks, demonstrating superior generalizability and label-efficient adaptation for clinical sleep analysis.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper12
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 被引用 24,064 次
- Emerging Properties in Self-Supervised Vision TransformersMathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou 等ICCV 2021 · 被引用 8,921 次
- FlashAttention-2: Faster Attention with Better Parallelism and Work PartitioningTri DaoICLR 2024 · 被引用 2,600 次
- On Layer Normalization in the Transformer ArchitectureRuibin Xiong, Yunchang Yang, Di He, Kai Zheng 等ICML 2020 · 被引用 1,388 次
- A decoder-only foundation model for time-series forecastingAbhimanyu Das, Weihao Kong, Rajat Sen, Yichen ZhouICML 2024 · 被引用 601 次
相关 Paper
- SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory SignalsRahul Thapa, Bryan He, Magnus Ruud Kjær, Hyatt E. Moore IV 等ICML 2024 · 被引用 48 次
- SleepLM: Natural-Language Intelligence for Human SleepZongzhe Xu, Zitao Shuai, Eideen Mozaffari, Ravi Aysola 等ICML 2026 · 被引用 10 次
- OSF: On Pre-training and Scaling of Sleep Foundation ModelsZitao Shuai, Zongzhe Xu, David Yang, Wei Wang 等ICML 2026 · 被引用 8 次
- sleep2vec: Unified Cross-Modal Alignment for Heterogeneous Nocturnal BiosignalsWeixuan Yuan, Zengrui Jin, Yichen Wang, Donglin Xie 等ICLR 2026 · 被引用 4 次
- Personalized Sleep Staging Leveraging Source-free Unsupervised Domain AdaptationYangxuan Zhou, Sha Zhao, Jiquan Wang, Haiteng Jiang 等AAAI 2025
