AlphaFold-Embed-8M

MedinaMemorySystems/AlphaFold-Embed-8M

Model Description

AlphaFold-Embed-8M is a compact yet highly effective protein sequence embedding model designed for structural confidence prediction, domain boundary analysis, and disorder assessment. Fine-tuned from facebook/esm2_t6_8M_UR50D, this model bridges sequence and structure by learning to predict structural features directly from primary amino acid sequences.

It uniquely provides per-residue pLDDT (predicted local distance difference test) confidence prediction, domain boundary detection, and intrinsic disorder scoring. Furthermore, it supports UniProt ID resolution and generates structural homology embeddings that are compatible with Foldseek.

Intended Uses

The primary use cases for AlphaFold-Embed-8M include:

• Predicting structural confidence (pLDDT) without generating a full 3D structure.
• Identifying protein domains and their boundaries from sequence alone.
• Assessing intrinsic disorder in protein sequences.
• Embedding proteins for fast, Foldseek-compatible structural homology searches.

Protein Sequence Representation

Sequences are represented directly as amino acid characters (e.g., "MKTII..."). The model uses standard ESM tokenization. The maximum sequence length is 1024 residues; longer proteins should be truncated or processed in overlapping chunks.

Architecture Details

AlphaFold-Embed-8M leverages the ESM-2 architecture, optimized for efficiency:

• Parameters: 8M
• Layers: 6
• Attention Heads: 20
• Hidden Dimension: 320
• Max Context Length: 1024 tokens
• Embeddings: Learned positional embeddings
• Extensions: Custom contact prediction head adapted for pLDDT regression

System Prompt / Prompting Template

AlphaFold-Embed-8M takes pure amino acid sequences. No special prompting template is required, but input strings must contain only valid amino acid character tokens.

Sequence format: <AMINO_ACID_STRING>
Example: MKTIIALSYIFCLVFADYKDDDDK

Quantization & Memory Footprint Table

Benchmark Results

AlphaFold-Embed-8M was evaluated on multiple structural biology benchmarks:

Python Usage Example (Transformers & Biopython)

You can use this model with the transformers library to extract per-residue embeddings and predict structural properties, seamlessly integrating with Biopython:

from transformers import AutoTokenizer, AutoModel
from Bio import SeqIO
import torch

# Load model and tokenizer
model_name = "MedinaMemorySystems/AlphaFold-Embed-8M"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

# Parse a sequence using Biopython
record = next(SeqIO.parse("protein.fasta", "fasta"))
protein_sequence = str(record.seq)

# Encode the protein sequence
inputs = tokenizer(protein_sequence, return_tensors="pt", truncation=True, max_length=1024)

with torch.no_grad():
    outputs = model(**inputs)

# Extract per-residue embeddings
embeddings = outputs.last_hidden_state
print(f"Embedding shape: {embeddings.shape}")

Ethics, Biosafety Note & Limitations

• Biosafety Note: This model's predictions of structure and disorder are predictive in nature. They should be rigorously validated experimentally when used in the design of novel proteins or therapeutics. Ensure adherence to institutional biosafety guidelines when predicting structures for pathogenic organisms.
• The maximum sequence length is limited to 1024 residues; longer proteins will need to be truncated or processed in overlapping chunks.
• While the model predicts structural confidence (pLDDT) accurately, it does not output physical 3D coordinates.
• Disorder predictions for extremely short peptides (< 20 amino acids) may exhibit higher variance.

Citation

@misc{medinamemorysystems2026alphafoldembed,
  title={AlphaFold-Embed-8M: Efficient Structural Feature Prediction from Protein Sequences},
  author={MedinaMemorySystems},
  year={2026},
  publisher={Hugging Face}
}