paraphrase-multilingual-MiniLM-L12-v2-latin

paraphrase-multilingual-MiniLM-L12-v2 — Latin-script vocabulary

Xenova/paraphrase-multilingual-MiniLM-L12-v2 (the ONNX export of sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) with its vocabulary cut from 250002 to 128507 pieces: every piece written in Latin script or script-neutral characters is kept; pieces containing CJK, Arabic, Devanagari, Thai, Cyrillic, Greek and other scripts are removed, from the tokenizer and from the model's embedding table alike. Nothing else changes — the transformer layers are byte-identical.

For text in any Latin-script language the output vectors are identical to the full model's (same segmentation, same rows), verified at cosine 1.0000 over 408 German, English, Italian, Spanish and French texts. Text in other scripts segments to single characters and degrades gracefully.

Why: the full model's 250k-piece vocabulary costs ~1 GB of memory in a mobile WebView (measured: an iPhone killed the host process at its 2 GB limit). This variant loads in ~370–400 MB. Built for the Obsidian plugin Pythia by scripts/prune-embedding-model.py, which reproduces it from the upstream files. Only the q8 (model_quantized.onnx) weights are published.

Licensed Apache-2.0 like the original; the original authors' work, credited above, is unchanged by this pruning.