static-nomic-384-pten-v2-st

This is the sentence-transformers compatible version of cnmoro/static-nomic-384-pten-v2.

It is numerically identical to the original (cosine similarity 1.0000000000, max absolute difference 1.5e-08); the only change is how the weights are stored. The original uses Model2Vec's vocabulary_quantization, which packs the vocabulary into a smaller shared table plus mapping and weights tensors. Sentence Transformers' StaticEmbedding cannot read that layout, so here the embedding matrix is materialized to one row per token.

Use cnmoro/static-nomic-384-pten-v2 if you want the smaller download (52 MB vs 424 MB) and are loading with model2vec.

This Model2Vec model was created by using Tokenlearn, with nomic-embed-text-v2-moe as a base.

The output dimension is 384.

The evaluation in the model card was executed using this distilled model, not the original.

This model was trained in streaming mode over large precomputed feature shards with incremental PCA (384d), vocabulary quantization capped at 32k effective tokens, and fine-tuning optimizations for large-scale data.

This is a better model than cnmoro/static-nomic-384-pten

Usage

Load this model using model2vec library:

from model2vec import StaticModel

model = StaticModel.from_pretrained("cnmoro/static-nomic-384-pten-v2")

# Compute text embeddings
embeddings = model.encode(["Example sentence"])

Or using sentence-transformers library:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer('cnmoro/static-nomic-384-pten-v2')

# Compute text embeddings
embeddings = model.encode(["Example sentence"])