turkish-bge-m3-model2vec

🇹🇷 Turkish BGE-M3 Model2Vec (533k Distilled Champion)

High-performance, ultra-fast Turkish static sentence embedding model distilled from BAAI/bge-m3 on 533k Turkish Wikipedia articles using 256-dimensional PCA teacher representations.

• Model Size: 19.36 MB (114x smaller than 2.2 GB BGE-M3)
• Embedding Dimension: 256
• Vocabulary Size: 39,655 pruned Turkish tokens
• Inference Speed: 63,012 sentences/sec on CPU (797x faster than BGE-M3)
• Accuracy (STSb-TR gorkem): 91.36% (Preserves ~95% of BGE-M3 teacher accuracy while beating the 182 MB unpruned model's 86.83% baseline)

---

📊 Benchmark Results

---

🚀 Quickstart & Usage

Installation

pip install model2vec

Inference

from model2vec import StaticModel

# Load model directly from Hugging Face
model = StaticModel.from_pretrained("altaidevorg/turkish-bge-m3-model2vec")

sentences = [
    "Yapay zeka modelleri doğal dil işlemede çığır açıyor.",
    "Türkiye'nin en kalabalık ve ekonomik merkezi İstanbul'dur.",
    "Hızlı ve hafif vektör modelleri edge cihazlarda çalışabilir."
]

embeddings = model.encode(sentences)
print(f"Shape: {embeddings.shape}")  # (3, 256)

---

🧠 Distillation Methodology

1. Vocabulary Pruning: Retains the 39,655 most frequent Turkish subwords from the multilingual BGE-M3 tokenizer.
2. Teacher Target Reduction: 1024d BGE-M3 teacher embeddings on 533k Turkish Wikipedia articles compressed to 256d via PCA (86.77% variance preserved).
3. Sentence-Level Weighted Mean Pooling: Eliminates segmentation drift via gradient descent distillation against normalized teacher targets.

---

📄 License

Apache-2.0