longevity-esm2-v4

Longevity Protein Classifier v6

Fine-tuned ESM-2 150M for binary classification of protein sequences as longevity-associated or not, trained on multi-species GenAge data with LoRA adapters.

Built as part of a personal ML learning arc — Week 3 of 8 — connecting protein language models to longevity biology.

---

Model Description

• Model type: ESM-2 150M + LoRA (r=16) sequence classifier
• Base model: facebook/esm2_t30_150M_UR50D
• Task: Binary classification — longevity-associated vs non-longevity
• Developed by: Mo Elzek
• License: Apache 2.0

---

Performance

---

Benchmark Results

---

Novel Predictions Not in GenAge

Proteins scoring above 0.50 that are not present in GenAge human database. These are the model's predictions of longevity-relevant proteins not yet catalogued — not validated findings.

---

Recommended Thresholds

Optimised threshold from val set: 0.06 (F1: 0.358)

The model produces a bimodal distribution — proteins it recognises score very high (above 0.50), proteins it does not score near zero. The flat recall curve from 0.05 to 0.70 reflects this — most longevity proteins are either clearly found or clearly missed.

---

Known Limitations — Read Before Use

1. Protein length truncation

Sequences longer than 512 amino acids are truncated from the C-terminus. This causes systematic failures on long proteins where the functional domain sits in the C-terminal half:

• MTOR (2,549 aa): kinase domain at residues 2181-2431 — truncated away
• TERT (1,132 aa): reverse transcriptase domain at 600-900 — truncated away

Do not use this model to score proteins above 800 amino acids without validating on known examples from that protein family first.

2. Family-specific blind spots

The model learned sirtuin and tumour suppressor sequence features well but has insufficient training examples to generalise to:

• Forkhead transcription factors (FOXO3 scores 0.000 despite being a canonical longevity gene and fitting within the 512 aa window)
• Large kinases (truncation compounds this)
• Telomerase complex proteins

3. Direction of effect not captured

The model cannot distinguish between:

• Pro-longevity proteins (overexpression extends lifespan)
• Anti-aging-disease proteins (loss of function accelerates aging)

Both may score high. A high score means "associated with longevity biology" not "activating this protein extends lifespan."

4. Not validated experimentally

Novel predictions are model outputs only. No wet lab validation has been performed. TFEB is the strongest prediction based on prior literature but this model did not discover TFEB — it independently ranked it highly, consistent with existing biology.

5. Not for clinical use

This is a research screening tool. Do not use for any clinical, diagnostic, or therapeutic decision-making.

---

Training Data

Positive set: GenAge database (genomics.senescence.info)

• Human GenAge: 306 human longevity-associated genes
• Model organism GenAge: Pro-Longevity genes only from 4 species
  • C. elegans: 283 genes
  • D. melanogaster: 125 genes
  • M. musculus: 85 genes
• Total positives: ~574

Negative set: Swiss-Prot reviewed proteins from same species

• Sampled proportionally per species (NEG_RATIO=10)
• Species weights applied: human 2.0x, mouse 1.5x, worm/fly 1.0x
• "Necessary for fitness" genes excluded from universe entirely
• Anti-Longevity genes excluded from positives

Filtering:

• Sequence length: 50-1500 amino acids
• Swiss-Prot reviewed only (manually curated)

---

Training Procedure

Architecture: ESM-2 150M + LoRA adapters

• LoRA rank: r=16, alpha=32, dropout=0.15
• Target modules: query, value attention projections
• Trainable parameters: ~4.7M of 150M total (3.1%)

Loss function: Focal loss with contrastive margin penalty

• gamma=1.0 (softer than standard gamma=2.0)
• Label smoothing=0.1
• Contrastive margin=0.30 (explicit separation penalty)
• Class weights: balanced

Optimiser: AdamW, lr=2e-4, weight_decay=0.01 Schedule: Cosine with warmup (10% warmup steps) Early stopping: Patience=4 on val AUPRC Best epoch: 10 of 20

Hardware: NVIDIA T4 16GB (Kaggle) Training time: ~2 hours

---

How to Use

from transformers import AutoTokenizer, EsmForSequenceClassification
from peft import PeftModel
import torch

# Load model
base = EsmForSequenceClassification.from_pretrained(
    "facebook/esm2_t30_150M_UR50D",
    num_labels=2,
    ignore_mismatched_sizes=True
)
model = PeftModel.from_pretrained(base, "YOUR_USERNAME/longevity-esm2-v6")
tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/longevity-esm2-v6")

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
model.eval()

def score_sequence(sequence, threshold=0.06):
    inputs = tokenizer(
        sequence,
        max_length=512,
        padding="max_length",
        truncation=True,
        return_tensors="pt"
    )
    with torch.no_grad():
        outputs = model(
            input_ids=inputs["input_ids"].to(device),
            attention_mask=inputs["attention_mask"].to(device)
        )
        prob = torch.softmax(outputs.logits, dim=1)[:, 1].item()
    return {
        "probability": round(prob, 4),
        "prediction": "Longevity" if prob >= threshold else "Non-longevity",
        "threshold": threshold,
        "warning": "Truncated to 512 aa" if len(sequence) > 512 else None
    }

# Example
result = score_sequence("MKTAYIAKQRQISFVK...")
print(result)

Recommended thresholds:

• 0.05-0.06 for screening (maximise recall)
• 0.50 for high-confidence hits only

---

Experiment History

This model is v6 in a series of iterative experiments:

---

Citation

If you use this model in research, please cite: @misc{elzek2026longevity, author = {Elzek, Mo}, title = {Longevity Protein Classifier: Multi-species ESM-2 Fine-tuning}, year = {2026}, publisher = {HuggingFace}, url = {https://huggingface.co/YOUR_USERNAME/longevity-esm2-v6} }

---

Contact

Built by Mo Elzek as part of the London Longevity Network ML project arc.
Feedback and collaboration welcome.