PlantPLM-150M

PlantPLM-150M

<img src="Plant_PLM_logo.png" alt="Alt Text" width="800">

ESM-2 150M parameter model continued-pretrained on Viridiplantae (plant) protein sequences.

This is a domain-adapted version of facebook/esm2_t30_150M_UR50D, fine-tuned on a non-redundant subset of UniProt TrEMBL plant-kingdom proteins.

Part of the Plant-PLM - ESM-2 models at 8M, 35M, 150M, and 650M parameters, each adapted on plant protein data.

---

Model Description

---

Training Data

Unlike the 8M and 35M variants (trained on the raw, redundant plant TrEMBL corpus), this model was trained on a redundancy-reduced ("nr50") corpus: the raw Viridiplantae corpus was clustered with MMseqs2 easy-linclust (50% identity / 80% coverage, mirroring ESM-2's own training-data construction) and one representative sequence per cluster was kept.

---

Training Details

Final metrics (validation set, 5% holdout):

---

Usage

from transformers import EsmForMaskedLM, EsmTokenizer
import torch

model = EsmForMaskedLM.from_pretrained("dipayan26/PlantPLM-150M")
tokenizer = EsmTokenizer.from_pretrained("dipayan26/PlantPLM-150M")

# --- Masked token prediction ---
sequence = "MSPQTETKASVGFKAGVKDYKLTYYTPEYETK"
inputs = tokenizer(sequence, return_tensors="pt")

# mask one position
inputs["input_ids"][0, 5] = tokenizer.mask_token_id

with torch.no_grad():
    logits = model(**inputs).logits

masked_pos = (inputs["input_ids"] == tokenizer.mask_token_id).nonzero()[0, 1]
top5 = logits[0, masked_pos].topk(5)
print(tokenizer.convert_ids_to_tokens(top5.indices.tolist()))

# --- Sequence embedding ([CLS] token) ---
inputs = tokenizer(sequence, return_tensors="pt")
with torch.no_grad():
    hidden = model.esm(**inputs).last_hidden_state
cls_embedding = hidden[0, 0, :]   # shape: [640]
print("Embedding shape:", cls_embedding.shape)

---

Intended Use

• Plant protein function prediction — GO term annotation, subcellular localization, signal peptide detection
• Plant-specific protein embeddings — clustering, retrieval, similarity search
• Transfer learning starting point — fine-tune on small labeled plant protein datasets

Out-of-scope Use

• Non-plant organisms — the model has been shifted toward Viridiplantae statistics; use the original facebook/esm2_t30_150M_UR50D for general protein tasks
• Structural prediction — not trained for structure; use ESMFold for that

---

Limitations

• No downstream benchmark evaluation has been run on this checkpoint yet

---

Citation

If you use this model, please cite:

@misc{sarkar2026plantplm,
  author       = {Sarkar, Dipayan},
  title        = {PlantPLM: Domain-Adaptive Pretraining of ESM-2 on Viridiplantae Proteins},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/dipayan26/PlantPLM-150M}},
}