preferential-250k

Preferential-250k

Preferential-250k is an antibody language model that uses an ESM-2 architecture. It was pre-trained on paired sequences from Jaffe et al. and Hurtado et al. Datasets used for pre-training are available on Zenodo and code is available on GitHub. More details can be found in our paper published in Patterns.

Use

Load the model and tokenizer as follows:

from transformers import EsmTokenizer, EsmForMaskedLM

model = EsmForMaskedLM.from_pretrained("brineylab/preferential-250k")
tokenizer = EsmTokenizer.from_pretrained("brineylab/preferential-250k")

The tokenizer expects sequences formatted as: HEAVY_CHAIN<cls><cls>LIGHT_CHAIN.

The model can be finetuned for classification tasks (such as specificity and pair classification in the paper) by loading the model with a sequence classification head:

from transformers import EsmForSequenceClassification

model = EsmForSequenceClassification.from_pretrained("brineylab/preferential-250k")

# freeze the base model weights prior to finetuning
for param in model.base_model.parameters():
    param.requires_grad = False