sleep2vec: Unified Cross-Modal Alignment for Heterogeneous Nocturnal Biosignals
Weixuan Yuan, Zengrui Jin, Yichen Wang, Donglin Xie, Ziyi Ye, Chao Zhang, Xuesong Chen
Abstract
Tasks ranging from sleep staging to clinical diagnosis traditionally rely on standard polysomnography (PSG) devices, bedside monitors and wearable devices, which capture diverse nocturnal biosignals (e.g., EEG, EOG, ECG, SpO). However, heterogeneity across devices and frequent sensor dropout pose significant challenges for unified modelling of these multimodal signals. We present sleep2vec, a foundation model for diverse and incomplete nocturnal biosignals that learns a shared representation via cross-modal alignment. sleep2vec is contrastively pre-trained on 42,249 overnight recordings spanning nine modalities using a Demography, Age, Site & History-aware InfoNCE objective that incorporates physiological and acquisition metadata (e.g., age, gender, recording site) to dynamically weight negatives and mitigate cohort-specific shortcuts. On downstream sleep staging and clinical outcome assessment, sleep2vec consistently outperforms strong baselines and remains robust to any subset of available modalities and sensor dropout. We further characterize, to our knowledge for the first time, scaling laws for nocturnal biosignals with respect to modality diversity and model capacity. Together, these results show that unified cross-modal alignment, coupled with principled scaling, enables label-efficient, general-purpose modelling of real-world nocturnal biosignals.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext f8a12ca0-c6eb-450b-8b96-f645f25e5d35Cited by top-tier papers1
Ask how each one uses itBuilds on8
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- Generalizable Sleep Staging via Multi-Level Domain AlignmentJiquan Wang, Sha Zhao, Haiteng Jiang, Shijian Li et al.AAAI 2024 · 28 citations
- Robust Sleep Staging over Incomplete Multimodal Physiological Signals via Contrastive ImaginationQi Shen, Junchang Xin, Bing Tian Dai, Shudi Zhang et al.NeurIPS 2024 · 15 citations
- SleepSMC: Ubiquitous Sleep Staging via Supervised Multimodal CoordinationShuo Ma, Yingwei Zhang, Yiqiang Chen, Hualei Wang et al.ICLR 2025
- ImageBind One Embedding Space to Bind Them AllRohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh et al.CVPR 2023
Related papers
- SleepMG: Multimodal Generalizable Sleep Staging with Inter-modal Balance of Classification and Domain DiscriminationShuo Ma, Yingwei Zhang, Qiqi Zhang, Yiqiang Chen et al.ACM MM 2024 · 1 citation
- ElaSleepNet: Exploring an Elastic Multimodal Neural Network for Sleep Staging via Temporal and Contextual Consistency LearningQi Shen, Junchang Xin, Bing Tian Dai, Shudi Zhang et al.ACM MM 2025
- SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory SignalsRahul Thapa, Bryan He, Magnus Ruud Kjær, Hyatt E. Moore IV et al.ICML 2024 · 48 citations
- OSF: On Pre-training and Scaling of Sleep Foundation ModelsZitao Shuai, Zongzhe Xu, David Yang, Wei Wang et al.ICML 2026 · 8 citations
- PATCHCODE: Discrete Latent Predictive Learning for EEG Foundation ModelKIEREN YU, Ziyang Liu, Chang Huang, Kaishun WUICML 2026
