sanskritgemma-256

sanskritgemma-256

A Model2Vec static distillation of Ganaraj/rgveda-embedding-gemma — an embeddinggemma variant tuned on Vedic Sanskrit. Static embeddings are a lookup table: no transformer runs at inference, so encoding is ~1,000× faster on CPU at the cost of word order and context.

• 256 dimensions, float16, PCA-256, mean pooling, normalized — ~131 MB
• Tokenizer: Ganaraj/rgveda-embedding-gemma
• Sibling widths at 384, 512 and 768 exist; see the Matryoshka note below before reaching for them

Usage

from model2vec import StaticModel

model = StaticModel.from_pretrained("karthikrajgopal/sanskritgemma-256")
embeddings = model.encode(["धर्मक्षेत्रे कुरुक्षेत्रे", "the field of dharma"])

Evaluation

Measured on the complete Bhagavad Gītā — 701 verses indexed as Devanagari only, queried by their IAST transliteration and by two independent published English translations. An English query is therefore genuine cross-lingual retrieval with no leakage. Scores are MRR@10.

Against other static models, encoder only

The teacher is the lever, not the width. Doubling the general embeddinggemma distillation to 512 dims made it worse; re-distilling from a Sanskrit-tuned teacher at 256 dims moved English-close retrieval from 0.251 to 0.317.

Matryoshka: the widths are nearly flat

256 reaches 98% of 768's quality at a third of the disk, which is why this is the width worth publishing.

When to use it, and when not to

This model is strongest where the encoder is doing the retrieval: flat chunk search, an ANN-only path, or a transliteration-heavy corpus — its IAST top-1 (0.487) is the best of the static models tested, against potion's 0.420.

Measured inside a full retrieval stack — tree structure, section roll-up and assembled context — potion-multilingual pulled ahead (0.238 against 0.213 mean MRR, paired bootstrap 95% CI [+0.016, +0.074] over 500 queries). Once whole sections are being assembled, what is left for the encoder is loose English semantics, and a broadly-trained multilingual model handles that better. Benchmark the configuration you ship, not the encoder alone.

Static pooling also cannot recover a word that sandhi or compounding has fused: dharmakṣetre is one token. In litesearch that gap is closed outside the encoder, by indexing lemmas and Monier-Williams glosses alongside the text.

Provenance

Distilled with Model2Vec. Full evaluation, including the paired significance test, is in litesearch's 10_sanskrit_eval notebook.

MIT, following the Model2Vec tooling. The teacher derives from Google's embeddinggemma, whose Gemma terms of use flow through — check those before redistributing derivatives.