Downloads · 30 days
0
azettaai/minilm-l6-yat-ffn-swap
minilm-l6-yat-ffn-swap is a machine learning model from azettaai. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This repo contains Yat student replacements for all 6 feed-forward blocks of sentence-transformers/all-MiniLM-L6-v2.
Downloads · 30 days
0
Access
Public
Updated May 30, 2026
Repo size
85 MB
Likes
0
Public
Click a slice to open those files.
.safetensors85 MB · 100%
From the Hugging Face model README
This repo contains Yat student replacements for all 6 feed-forward blocks of sentence-transformers/all-MiniLM-L6-v2.
The checkpoints were produced on Kaggle with a three-phase pipeline:
The published model is a lightweight patch over the base MiniLM model. Loader code in yat_minilm.py downloads the base model, loads the Phase-2 Yat checkpoints, and replaces every BERT feed_forward_chunk.
Phase 1 mean rho: 0.005847
Phase 2 mean rho: 0.098501 -> 0.001715
MTEB STS scores:
| Task | Baseline | Yat-swapped |
|---|---|---|
| STSBenchmark | 0.820325 | 0.816818 |
| STS12 | 0.723690 | 0.720878 |
| STS16 | 0.789895 | 0.789110 |
from yat_minilm import load_model
model = load_model("azettaai/minilm-l6-yat-ffn-swap")
emb = model.encode(["hello world", "yat swapped minilm"])
print(emb.shape)
phase2/block0.safetensors ... phase2/block5.safetensors: final Yat FFN replacements.phase1/: random/on-shell warm-start checkpoints.scripts/: Kaggle scripts used to train and evaluate the model.yat_minilm.py: loader and patching code.