Downloads · 30 days
32
22% of all-time downloads
AmelieSchreiber/esm2_t12_35M_ptm_lora_2100K
esm2_t12_35M_ptm_lora_2100K is a token classification model from AmelieSchreiber. Use it when you need labels on individual words, such as names. It is set up for peft. The card lists the license as mit.
This is a LoRA finetuned version of esm2t1235MUR50D for predicting post translational modification sites.
Downloads · 30 days
32
22% of all-time downloads
All-time downloads
148
Public
Repo size
968 KB
Likes
0
Public
Click a slice to open those files.
.pt633 KB · 58%
From the Hugging Face model README
This is a LoRA finetuned version of esm2_t12_35M_UR50D for predicting post translational modification sites.
"eval_loss": 0.4661065936088562,
"eval_accuracy": 0.9876599555715365,
"eval_auc": 0.8673592596422711,
"eval_precision": 0.14941997670219148,
"eval_recall": 0.7463955099754822
"eval_f1": 0.24899413187145658,
"eval_mcc": 0.3305508498121041,
To use this model, run the following:
!pip install transformers -q
!pip install peft -q
from transformers import AutoModelForTokenClassification, AutoTokenizer
from peft import PeftModel
import torch
# Path to the saved LoRA model
model_path = "AmelieSchreiber/esm2_t12_35M_ptm_lora_2100K"
# ESM2 base model
base_model_path = "facebook/esm2_t12_35M_UR50D"
# Load the model
base_model = AutoModelForTokenClassification.from_pretrained(base_model_path)
loaded_model = PeftModel.from_pretrained(base_model, model_path)
# Ensure the model is in evaluation mode
loaded_model.eval()
# Load the tokenizer
loaded_tokenizer = AutoTokenizer.from_pretrained(base_model_path)
# Protein sequence for inference
protein_sequence = "MAVPETRPNHTIYINNLNEKIKKDELKKSLHAIFSRFGQILDILVSRSLKMRGQAFVIFKEVSSATNALRSMQGFPFYDKPMRIQYAKTDSDIIAKMKGT" # Replace with your actual sequence
# Tokenize the sequence
inputs = loaded_tokenizer(protein_sequence, return_tensors="pt", truncation=True, max_length=1024, padding='max_length')
# Run the model
with torch.no_grad():
logits = loaded_model(**inputs).logits
# Get predictions
tokens = loaded_tokenizer.convert_ids_to_tokens(inputs["input_ids"][0]) # Convert input ids back to tokens
predictions = torch.argmax(logits, dim=2)
# Define labels
id2label = {
0: "No ptm site",
1: "ptm site"
}
# Print the predicted labels for each token
for token, prediction in zip(tokens, predictions[0].numpy()):
if token not in ['<pad>', '<cls>', '<eos>']:
print((token, id2label[prediction]))