Downloads · 30 days
0
MethylLlama
netanelazran1/MethylLlama
MethylLlama is a machine learning model from netanelazran1. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as apache-2.0.
Weights of MethylLlama, a Transformer encoder for DNA methylation array profiles pretrained with the Whole-CpG Encoder–Decoder (WCED) objective, and of the five chronological-age models fine-tuned from it. Code, token…
Access
Public
Updated Oct 18, 2026
Repo size
780 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.ckpt451 MB · 100%
At a glance
- Library
- pytorch
- License
- apache-2.0
- Access
- Public
- Created
- Jun 22, 2026
- Updated
- Oct 18, 2026
- SHA
- 350dd50d
Model card
From the Hugging Face model README
MethylLlama
Weights of MethylLlama, a Transformer encoder for DNA methylation array profiles pretrained with the Whole-CpG Encoder–Decoder (WCED) objective, and of the five chronological-age models fine-tuned from it. Code, tokenizer and instructions: github.com/netanelazran11/MethylLlama.
Files
| File | Content | sha256 (prefix) |
|---|---|---|
pretrained/methyllama_pretrained_epoch85.ckpt | WCED-pretrained encoder, epoch 85 (lowest validation reconstruction loss) | c9e6202d3358b6cc |
finetuned_age/fold0.ckpt … fold4.ckpt | age models of the five cross-validation folds | see SHA256SUMS |
resources/cpg_genomic_rank.npy | genomic-rank position ids, 49,156-CpG pretraining panel | 082fe5a3141a14e8 |
resources/cpg_genomic_rank_finetune.npy | genomic-rank position ids, 21,368-CpG downstream panel | fe0b03ba4f67cf6e |
tokenizer/cpg_sites/ | the 49,161-entry CpG vocabulary | |
SHA256SUMS | full checksums of the files above | |
ORIGINAL_SHA256SUMS | checksums of the checkpoints as written by the training runs |
The checkpoints contain the weights and hyper-parameters of the training runs' checkpoint files
(PyTorch Lightning format); optimizer, scheduler and callback states were removed. Every tensor is
identical to the original file, and the models loaded from both are identical (checksums of the
originals in ORIGINAL_SHA256SUMS). The files hold Python objects besides tensors, so they are
loaded with weights_only=False; the package does this on import.
Model
6-layer pre-norm Transformer encoder (hidden size 256, 4 heads, SwiGLU 512, RMSNorm), rotary position embeddings driven by each CpG's genomic rank, 16.6 M parameters. Sample representation: pooled CLS output. Age head: 256 → 128 → 64 → 1.
Use
git clone https://github.com/netanelazran11/MethylLlama && cd MethylLlama
pip install -e ".[tutorials]"
bash scripts/download_checkpoint.sh # writes checkpoints/ in this layout
See the repository README and tutorials/ for encoding profiles and predicting age. Inputs must be
passed with genomic-rank position ids from the resources/ file that matches their CpG panel.
Citation
To be added with the preprint.
- Library
- pytorch
- License
- apache-2.0
- Created
- Jun 22, 2026
- Updated
- Oct 18, 2026