TolerantECG

TolerantECG: A Foundation Model for Imperfect Electrocardiogram

Official HuggingFace checkpoint for TolerantECG: A Foundation Model for Imperfect Electrocardiogram, accepted at the 33rd ACM International Conference on Multimedia (ACM MM 2025).

• Repository: GitHub
• Paper: arXiv:2507.09887 / DOI: 10.1145/3746027.3755287

---

💡 Overview

Electrocardiogram (ECG) data in clinical practice frequently suffers from noise, baseline wander, electrode motion artifacts, and missing or corrupted leads. TolerantECG is a foundation model designed specifically to handle imperfect ECG signals by learning robust representations across signal perturbations.

TolerantECG unifies:

1. Self-Supervised Learning (DINO) with signal masking and synthetic/real noise perturbations (baseline wander, electromyogram, electrode motion).
2. Contrastive Language-ECG Pretraining (CLIP) paired with clinical reports via BioLinkBERT (michiyasunaga/BioLinkBERT-base).

The primary backbone feature extractor is a 1D ConvNeXt V2 encoder producing 768-dimensional representations per 12-lead ECG input.

---

🏗️ Model Architecture

• ECG Backbone: ConvNeXt V2 (1D 12-lead configuration)
• Input Shape: (batch_size, 12, length) (e.g., 12-lead ECG signals sampled at 500 Hz for 10 seconds → (B, 12, 5000))
• Embedding Dimension: 768
• Pre-trained Weights: Provided as PyTorch state_dict (.pth / .pt) for the ConvNeXt V2 encoder backbone.

---

💻 Quick Start & Usage

You can download the pre-trained weights directly using huggingface_hub and load them into the ConvNeXtV2 backbone using PyTorch.

Installation

pip install torch huggingface_hub

Loading the Model & Extracting Embeddings

import torch
from huggingface_hub import hf_hub_download
from src.models.ecg_encoder.convnext import ConvNeXtV2

# 1. Instantiate the ConvNeXt V2 ECG Encoder (12-lead input, 768-dim output)
model = ConvNeXtV2(
    in_chans=12,
    depths=[3, 3, 9, 3],
    dims=[96, 192, 384, 768],
    drop_path_rate=0.0
)

# 2. Download pre-trained weights from HuggingFace Hub
weights_path = hf_hub_download(
    repo_id="ndhuynh02/TolerantECG",
    filename="TolerantECG_encoder.pth"
)

# 3. Load state_dict into the model
state_dict = torch.load(weights_path, map_location="cpu")
model.load_state_dict(state_dict)
model.eval()

# 4. Extract embeddings from sample 12-lead ECG tensor (Batch size=2, 12 leads, 5000 time steps)
dummy_ecg = torch.randn(2, 12, 5000)

with torch.no_grad():
    # Returns 768-dimensional embedding vector per sample
    embeddings = model(dummy_ecg)

print("ECG Embeddings Shape:", embeddings.shape)
# Output: torch.Size([2, 768])

---

📊 Training & Datasets

TolerantECG was pre-trained and evaluated on large-scale public ECG benchmarks:

---

📄 License

This model checkpoint and repository are distributed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0). Refer to the LICENCE file for details.

---

📝 Citation

If you use TolerantECG in your research, please cite our ACM MM 2025 paper:

@inproceedings{10.1145/3746027.3755287,
  author    = {Nguyen, Huynh Dang and Pham, Trong-Thang and Le, Ngan and Nguyen, Van},
  title     = {TolerantECG: A Foundation Model for Imperfect Electrocardiogram},
  year      = {2025},
  isbn      = {9798400720352},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  url       = {https://doi.org/10.1145/3746027.3755287},
  doi       = {10.1145/3746027.3755287},
  booktitle = {Proceedings of the 33rd ACM International Conference on Multimedia},
  pages     = {8097–8105},
  numpages  = {9},
  keywords  = {contrastive learning, electrocardiogram (ecg), foundation model, imperfect signal, knowledge retrieval, self-supervised learning},
  location  = {Dublin, Ireland},
  series    = {MM '25}
}