Byrne-Embed

<img src="Byrne-LLMs.png" width="50%">

Byrne-Embed

85M sentence embedder. Maps text to 768-dim unit-norm vectors - similarity, retrieval, clustering, reranking.

Backbone is a SpikeWhale decoder (Byrne line). Mean-pool last hidden state, learned head to 768, unit-normalize. Cosine is a dot product.

Benchmark vs. EmbeddingGemma-300M

I ran Byrne-Embed against Google's EmbeddingGemma-300M on 4,000 held-out sentences - educational web, encyclopedic, instruction/chat. Geometry tracks EmbeddingGemma at about 1/3.5 the params:

The two that matter most - do they agree on which sentences are similar - are Pearson 0.97 / Spearman 0.96. When EmbeddingGemma says two sentences are similar, this one says the same thing. 94.7% of sentences sit within 0.90 cosine. Top-1 retrieval is lower and that is expected, not a quality hole: dense real sentences have a lot of near-ties (0.88 vs 0.87), so #1 flips between near-duplicates. That is why Recall@10 stays ~78% and neighbour-ranking correlation is 0.95. Same neighbourhood; they just swap rank 1 and 2 on near-identical candidates.

Reproduce with run_tests.py (loads both models, prints the full table).

MTEB English Benchmark - MTEB(eng, v2)

Evaluated with the official mteb library on the full MTEB(eng, v2) suite (41/41 tasks). Raw results are in mteb_results/; machine-readable scores are in the model-index metadata above.

Overall MTEB(eng, v2) mean: 50.79

STS

Classification

PairClassification

Clustering

Reranking

Retrieval

Summarization

Usage

Standard transformers via trust_remote_code (projection head is fused; one from_pretrained loads everything):

import torch
from transformers import AutoModel, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Quazim0t0/Byrne-Embed", trust_remote_code=True)
model = AutoModel.from_pretrained("Quazim0t0/Byrne-Embed", trust_remote_code=True).eval()

texts = ["The cat sat on the windowsill.", "A feline rested by the window."]
enc = tok(texts, return_tensors="pt", padding=True, truncation=True, max_length=128)
with torch.no_grad():
    emb = model(**enc).last_hidden_state          # (2, 768), L2-normalized

print(float(emb[0] @ emb[1]))                      # cosine similarity ~ 0.83

forward() returns L2-normalized 768-dim sentence embeddings. Cosine is a dot product.

Files

Limitations

• English-centric evaluation; non-English is untested.
• Weak spot I saw: finance/economics paraphrase retrieval. General semantic similarity is strong.
• Custom architecture: load via the bundled byrne_embedder.py (local modeling code - no remote code execution).

Citation

If you use Byrne-Embed, please cite:

@misc{byrne2026byrneembed,
  title        = {Byrne-Embed: A Compact 85M Sentence-Embedding Model},
  author       = {Byrne, Dean},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Quazim0t0/Byrne-Embed}},
}

License

Apache-2.0.