Lune

EMNLP Findings2025

SuPreME: A Supervised Pre-training Framework for Multimodal ECG Representation Learning

Mingsheng Cai, Jiuming Jiang, Wenhao Huang, Che Liu, Rossella Arcucci

Abstract

Cardiovascular diseases are a leading cause of death and disability worldwide. Electrocardiogram (ECG) is critical for diagnosing and monitoring cardiac health, but obtaining large-scale annotated ECG datasets is laborintensive and time-consuming. Recent ECG Self-Supervised Learning (eSSL) methods mitigate this by learning features without extensive labels but fail to capture fine-grained clinical semantics and require extensive task-specific fine-tuning. To address these challenges, we propose SuPreME, a Supervised Pre-training framework for Multimodal ECG representation learning. SuPreME is pre-trained using structured diagnostic labels derived from ECG report entities through a one-time offline extraction with Large Language Models (LLMs), which help denoise, standardize cardiac concepts, and improve clinical representation learning. By fusing ECG signals with textual cardiac queries instead of fixed labels, SuPreME enables zero-shot classification of unseen conditions without further fine-tuning. We evaluate SuPreME on six downstream datasets covering 106 cardiac conditions, achieving superior zero-shot AUC performance of 77.20%, surpassing state-of-the-art eSSLs by 4.98% 1 . Results demonstrate SuPreME's effectiveness in leveraging structured, clinically relevant knowledge for high-quality ECG representations.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale cites A Simple Framework for Contrastive Learning of Visual Representations
  • Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning cites A Simple Framework for Contrastive Learning of Visual Representations
  • An Empirical Study of Training Self-Supervised Vision Transformers cites A Simple Framework for Contrastive Learning of Visual Representations
  • An Empirical Study of Training Self-Supervised Vision Transformers cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • An Empirical Study of Training Self-Supervised Vision Transformers cites Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning
  • Barlow Twins: Self-Supervised Learning via Redundancy Reduction cites A Simple Framework for Contrastive Learning of Visual Representations
  • Barlow Twins: Self-Supervised Learning via Redundancy Reduction cites Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on17

  1. A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
  2. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  3. Bootstrap Your Own Latent - A New Approach to Self-Supervised LearningJean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec et al.NeurIPS 2020 · 9,171 citations
  4. Barlow Twins: Self-Supervised Learning via Redundancy ReductionJure Zbontar, Li Jing, Ishan Misra, Yann LeCun et al.ICML 2021 · 2,942 citations
  5. An Empirical Study of Training Self-Supervised Vision TransformersXinlei Chen, Saining Xie, Kaiming HeICCV 2021 · 2,340 citations

Related papers

  1. Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge EnhancementChe Liu, Zhongwei Wan, Cheng Ouyang, Anand Shah et al.ICML 2024 · 83 citations
  2. From Token to Rhythm: A Multi-Scale Approach for ECG-Language PretrainingFuying Wang, Jiacheng Xu, Lequan YuICML 2025
  3. Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language ModelJiarui Jin, Haoyu Wang, Hongyan Li, Jun Li et al.ICLR 2025
  4. Tracing the Heart's Pathways: ECG Representation Learning from a Cardiac Conduction PerspectiveTan Pan, Yixuan Sun, Chen Jiang, Qiong Gao et al.AAAI 2026
  5. Knowledge-enhanced Multimodal ECG Representation Learning with Arbitrary-Lead InputsChe Liu, Cheng Ouyang, Zhongwei Wan, Haozhe Wang et al.EMNLP Findings 2025 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines