trynmini-v2-static-7m

TrynMini v2 — static sentence embedder (7.7M params · 7.8 MB · no GPU)

⚠️ Superseded by LNTTushar/trynmini-v2-static-7m-v2

A stronger rebuild of this exact architecture scores 0.700 / 0.710 / 0.715 (STSB-dev, dim 64/128/256) vs 0.636 / 0.653 / 0.666 here. For new projects, use the v2 model — same tiny footprint and API, better vectors. This page is kept for provenance.

A static sentence embedding model: no transformer at inference. Encoding is tokenize → table lookup → SIF/Zipf-weighted mean pool → residual → L2-norm, so it runs on CPU with only numpy + tokenizers and ships as a single ~7.8 MB file. Supports Matryoshka truncation to 64 / 128 / 256 dims.

Benchmarks — STSBenchmark dev (Spearman)

This build used an earlier, weaker distillation pipeline than the v2 rebuild (which adds a properly trained BGE-distilled table plus an MNRL + Matryoshka residual). If you need the better scores, see trynmini-v2-static-7m-v2.

Install

pip install -U huggingface_hub numpy tokenizers safetensors

Quickstart

import sys; from huggingface_hub import snapshot_download
d = snapshot_download("LNTTushar/trynmini-v2-static-7m"); sys.path.insert(0, d)
from modeling_trynmini import TrynMiniV2
m = TrynMiniV2.from_pretrained(d)
emb = m.encode(["a man plays guitar", "someone plays a guitar"], dim=256)
print("cosine similarity:", float(emb[0] @ emb[1]))

Specs

• Params: ~7.7M (30,000 × 256 table + 256² residual)
• Deployment size: ~7.8 MB (per-row int8 table + tokenizer)
• Tokenizer: 30k WordPiece
• Runtime: numpy, tokenizers, safetensors — no PyTorch, no GPU

License

Apache-2.0.