SleepMaMi: A Universal Sleep Foundation Model for Integrating Macro- and Micro-structures
Keondo Park, Younghoon Na, Yourim Choi, Hyunwoo Ryu, Hyun-Woo Shin, Hyung-Sin Kim
Abstract
While the shift toward unified foundation models has revolutionized many deep learning domains, sleep medicine remains largely restricted to taskspecific models that focus on localized microstructure features. These approaches often neglect the rich, multi-modal context of Polysomnography (PSG) and fail to capture the global macrostructure of a full night's sleep. To address this, we introduce SleepMaMi, a Sleep Foundation Model engineered to master both hour-long sleep architectures and fine-grained signal morphologies. Our framework utilizes a hierarchical dual-encoder design: a Macro-Encoder to model full-night temporal dependencies and a Micro-Encoder to capture short-term characteristics from biosignals. Macro-Encoder is trained via Demographic-Guided Contrastive Learning, which aligns overnight sleep patterns with objective subject metadata, such as age, sex, and BMI to refine global representations. Micro-Encoder is optimized via a hybrid Masked Autoencoder (MAE) and multi-modal contrastive objective. Pre-trained on a massive corpus of >20,000 PSG recordings (158K hours), Sleep-MaMi outperforms or matches state-of-the-art foundation models across a diverse suite of downstream tasks, demonstrating superior generalizability and label-efficient adaptation for clinical sleep analysis.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Builds on12
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- Emerging Properties in Self-Supervised Vision TransformersMathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou et al.ICCV 2021 · 8,921 citations
- FlashAttention-2: Faster Attention with Better Parallelism and Work PartitioningTri DaoICLR 2024 · 2,600 citations
- On Layer Normalization in the Transformer ArchitectureRuibin Xiong, Yunchang Yang, Di He, Kai Zheng et al.ICML 2020 · 1,388 citations
- A decoder-only foundation model for time-series forecastingAbhimanyu Das, Weihao Kong, Rajat Sen, Yichen ZhouICML 2024 · 601 citations
Related papers
- SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory SignalsRahul Thapa, Bryan He, Magnus Ruud Kjær, Hyatt E. Moore IV et al.ICML 2024 · 48 citations
- SleepLM: Natural-Language Intelligence for Human SleepZongzhe Xu, Zitao Shuai, Eideen Mozaffari, Ravi Aysola et al.ICML 2026 · 10 citations
- OSF: On Pre-training and Scaling of Sleep Foundation ModelsZitao Shuai, Zongzhe Xu, David Yang, Wei Wang et al.ICML 2026 · 8 citations
- sleep2vec: Unified Cross-Modal Alignment for Heterogeneous Nocturnal BiosignalsWeixuan Yuan, Zengrui Jin, Yichen Wang, Donglin Xie et al.ICLR 2026 · 4 citations
- Personalized Sleep Staging Leveraging Source-free Unsupervised Domain AdaptationYangxuan Zhou, Sha Zhao, Jiquan Wang, Haiteng Jiang et al.AAAI 2025
