ProtSent-V2-ESMC-300M

ProtSent-V2 ESM-C 300M

Contrastively fine-tuned ESM-C 300M producing fixed-length protein embeddings where biological similarity maps to embedding proximity. Intended for retrieval, clustering, and nearest-neighbour transfer.

This is the ProtSent V2 recipe applied to an ESM-C backbone. Continued training from the Synthyra/ESMplusplus_small (ESMC-300M) weights.

Usage

from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim

model = SentenceTransformer("GrimSqueaker/ProtSent-V2-ESMC-300M", trust_remote_code=True)
emb = model.encode([
    "MKTLLLTLVVVTIVCLDLGYT",
    "MKTLLLTLVVVTIVCLDLGYN",
    "AGWYRSPQEGLKPVDTFKDIV",
])
print(cos_sim(emb[0], emb[1:]))

Embeddings are mean-pooled over the final layer, dimension 960. No Matryoshka heads on this release (see Training configuration).

Training data

Four sources. Pfam/AFDB/STRING are decontaminated against benchmark test sets (see below); the DMS source is not — see Limitations.

Pfam/AFDB pairs are drawn with the disjoint-pair sampler at up to 10 pairs per cluster (k=10), which spends the pair budget on distinct sequence pairs rather than repeating members — covers up to 90 distinct sequences per cluster instead of 10. STRING is a fixed 15M-pair subsample (seed 42).

Decontamination

Pfam, AlphaFold DB and STRING were searched against the benchmark test sequences with MMseqs2 easy-search (40% identity, 80% coverage, --cov-mode 1) and matching sequences removed before training — the same protsent-data-dc40 corpus used by the ProtSent-V2.5 ESM-2 releases.

Training configuration

Training code: github.com/oriel9p/ProtSent, train_esmc_300m_v2.sh.

Results

23-task suite, -e test, seed 42, both kNN and linear-probe transfer. Full per-task CSVs: results/benchmarks/ism/protsent_esmc_300m_v2_{knn,linear}/ in the repo above.

SCOPe-40 structural retrieval, eligible queries only

23-task suite vs the model's own untrained backbone and vs ISM-C

Win/tie/loss, tie tolerance 0.005:

Citation

Paper: ProtSent: Protein Sentence Transformers