bertmhc

BERTMHC MHC class II binding model

This repository contains a locally trained, paper-aligned BERTMHC checkpoint for MHC class II peptide-binding affinity prediction. It is intended for academic and non-profit non-commercial research.

Release scope: this is one fold and one random seed, not the complete five-fold, three-initialization-per-fold experiment from the BERTMHC paper. It is not a clinical model and does not predict antigen presentation, T-cell response, or immunogenicity.

Model description

The checkpoint fine-tunes the TAPE ProteinBERT base model on fold 1 of the Jensen et al. MHC class II binding dataset used by NetMHCIIpan-3.2 and BERTMHC. It uses:

• a 34-residue MHC class II pseudosequence;
• peptide padding or end-truncation to 24 residues;
• TAPE ProteinBERT with 12 layers, 12 attention heads, and hidden size 768;
• mean pooling over non-padding token representations;
• a two-layer MLP with hidden size 512;
• end-to-end fine-tuning with mean squared error on normalized affinity; and
• the first output logit, transformed with a sigmoid, as the binding score.

The normalized target is

y = 1 - log(IC50) / log(50000)

where IC50 is expressed in nM. A 500 nM binder threshold corresponds to a normalized target of approximately 0.426.

Evaluation

The best checkpoint was selected at epoch 10 using fold-1 evaluation auROC. The evaluation partition contains 27,332 measurements, including 11,013 binders at the 500 nM-derived threshold.

Fold 1 controlled learning-rate scheduling, checkpoint selection, and early stopping. These numbers are therefore validation results, not an independent test result. The BERTMHC paper reported aggregate five-fold performance of auROC 0.8822 and Pearson R 0.759 using three model initializations per fold. The two result sets are not directly equivalent.

Files

Loading the state dictionary

This is a custom PyTorch/TAPE checkpoint, not a Hugging Face Transformers AutoModel repository. The repository intentionally does not redistribute the official or locally modified BERTMHC source code. Researchers must obtain the official BERTMHC software separately, review and accept its non-commercial license, and use a runtime that implements the paper-specified masked mean pooling behavior.

After installing an authorized BERTMHC runtime, the state dictionary can be loaded as follows:

from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from tape import ProteinBertConfig

from bertmhc.bertmhc import BERTMHC

repo_id = "zmactep/bertmhc"
config_path = hf_hub_download(repo_id, "bertmhc_config.json")
weights_path = hf_hub_download(repo_id, "model.safetensors")

config = ProteinBertConfig.from_pretrained(config_path)
model = BERTMHC(config)
model.load_state_dict(load_file(weights_path), strict=True)
model.eval()

The released upstream BERTMHC implementation mean-pools all encoded positions, whereas the paper specifies masking padding positions before pooling. This checkpoint was trained with masked mean pooling. Using it with unmasked pooling will not reproduce the reported scores.

For input preparation, follow the official BERTMHC binding schema and tokenization. Each row needs an allele label, its 34-residue MHC pseudosequence, and a peptide. The mapping files in this repository are provided for convenience and provenance.

Intended use

Suitable uses include:

• non-commercial academic investigation of MHC-II peptide binding;
• replication and method-comparison studies;
• research-only ranking of peptide candidates; and
• evaluation as one component of an MHC-II ensemble.

Out-of-scope use

Do not use this checkpoint as:

• a clinical, diagnostic, or therapeutic decision system;
• a standalone predictor of antigen presentation or immunogenicity;
• evidence of efficacy or safety of a vaccine candidate;
• a calibrated probability of biochemical binding; or
• a commercial model or service.

Limitations

• Only fold 1 and seed 1111 were trained.
• The published 15-model cross-validation design has not been completed.
• The authors' independent 2,413-pair benchmark has not yet been evaluated.
• No local leave-one-molecule-out experiment was performed.
• Peptides longer than 24 residues are truncated at the end.
• Source duplicates and repeated peptide-allele measurements were preserved.
• Apple MPS runs were stable to reported precision but not proven bitwise deterministic across hardware and software versions.
• Scores have not been independently calibrated across alleles.

Training data

The model was trained from the official Jensen/NetMHCIIpan-3.2 binding folds:

• fold-1 training: 106,949 measurements;
• fold-1 evaluation: 27,332 measurements;
• complete source dataset: 134,281 measurements;
• complete source coverage: 80 MHC-II molecules, including 72 human and 8 murine molecules; and
• observed peptide lengths: 9-37 residues.

No training rows are redistributed in this model repository. Dataset source and processed-artifact hashes are recorded in training_metadata.json.

Paper and code alignment

The peer-reviewed paper specifies MSE regression, Pearson correlation, and padding-aware mean pooling for the binding task. The released BERTMHC code uses binary cross-entropy for the continuous binding target, computes Kendall tau as val_cor, and averages over padding positions. This checkpoint follows the paper for these three choices. GitHub issue 8 records the unresolved public question about the loss and metric discrepancy.

Licensing and attribution

The model weights are released under CC BY-NC 4.0; see MODEL_LICENSE.md. The official BERTMHC software is not included and remains subject to its separate NEC Laboratories Europe non-commercial research license. TAPE notices are reproduced in THIRD_PARTY_NOTICES.md.

Publications using this checkpoint must cite the BERTMHC and TAPE papers and must explicitly acknowledge:

The BERTMHC software was developed by NEC Laboratories Europe GmbH.

Citation

@article{cheng2021bertmhc,
  title = {BERTMHC: improved MHC-peptide class II interaction prediction with transformer and multiple instance learning},
  author = {Cheng, Jun and Bendjama, Ka{\"i}dre and Rittner, Karola and Malone, Brandon},
  journal = {Bioinformatics},
  volume = {37},
  number = {22},
  pages = {4172--4179},
  year = {2021},
  doi = {10.1093/bioinformatics/btab422}
}

@inproceedings{rao2019tape,
  title = {Evaluating Protein Transfer Learning with TAPE},
  author = {Rao, Roshan and Bhattacharya, Nicholas and Thomas, Neil and Duan, Yan and Chen, Xi and Canny, John and Abbeel, Pieter and Song, Yun S.},
  booktitle = {Advances in Neural Information Processing Systems},
  volume = {32},
  year = {2019}
}

Sources

• BERTMHC paper: https://doi.org/10.1093/bioinformatics/btab422
• Official BERTMHC repository: https://github.com/s6juncheng/BERTMHC
• Paper/code discrepancy issue: https://github.com/s6juncheng/BERTMHC/issues/8
• Jensen et al. dataset paper: https://doi.org/10.1111/imm.12889
• TAPE paper and code: https://pmc.ncbi.nlm.nih.gov/articles/PMC7774645/
• NetMHCIIpan-3.2 data page: https://services.healthtech.dtu.dk/services/NetMHCIIpan-3.2/