Winzling-Embed-a8m-64k

Winzling-Embed-a8m-64k

A 29.1 MiB UINT4 ONNX German/English/Russian text-embedding model derived from hotchpotch/bekko-embedding-v1-a8m.

Winzling-Embed-a8m-64k keeps Bekko a8m's complete 4-layer, 384-dimensional encoder while shrinking its multilingual vocabulary from 256,000 to 65,536 tokens selected for German (de), English (en) and Russian (ru). The transformer is then exported with block-wise asymmetric UINT4 weights; the token embedding table remains row-wise INT8.

The resulting ONNX graph is 29.1 MiB on disk and 26.4 MiB with zstd -19, while retaining the original model's 7,671,168 active transformer parameters.

On the fixed 10-task de/en/ru MTEB regression suite documented below, the model scores 54.979, versus 55.609 for the same 64k-vocabulary backbone before UINT4 quantization and 61.887 for the original 256k-vocabulary Bekko a8m.

[!IMPORTANT] 54.979 is not the official full MMTEB Multilingual v2 score. It is the mean of the fixed 10-task, de/en/ru regression suite documented in this model card. The upstream Bekko model reports 56.7 Mean(Task) on the official 131-task MMTEB Multilingual v2 benchmark. Those two aggregates are not directly comparable.

Highlights

• 29.1 MiB runtime ONNX graph
• 26.4 MiB as onnx/model_uint4.onnx.zst with Zstandard level 19
• 32,836,992 total parameters, down from 105,975,168
• 7,671,168 active transformer parameters remain unchanged
• 65,536-token vocabulary specialized for de, en, ru
• 4-layer mmBERT / ModernBERT-style encoder
• 384-dimensional sentence embeddings
• Mean pooling
• Cosine similarity
• Up to 8,192 input tokens inherited from Bekko
• No query/document prefixes
• Transformer MatMul weights: asymmetric UINT4, block size 32
• Token embedding table: row-wise INT8
• Export validation cosine vs trimmed FP32:
  • mean: 0.989615
  • worst: 0.986063
• Fixed compact MTEB regression mean:
  • original Bekko: 61.887
  • 64k FP32: 55.609
  • 64k UINT4: 54.979

What the name means

Winzling is German for a tiny little thing.

• Embed — dense text embedding model
• a8m — inherited Bekko configuration with ~8M active transformer parameters
• 64k — 65,536-token vocabulary size

64k does not refer to context length or embedding dimension.

Model details

The parameter accounting is particularly simple:

Original token table:
256,000 × 384 = 98,304,000 parameters

Trimmed token table:
65,536 × 384 = 25,165,824 parameters

Transformer / active parameters:
7,671,168 parameters

Trimmed total:
25,165,824 + 7,671,168 = 32,836,992 parameters

The encoder itself is not pruned. The size reduction comes almost entirely from removing vocabulary rows that are unnecessary for the selected language set.

Model lineage

mmBERT-small
    ↓ structural pruning
hotchpotch/mmBERT-L4H384-pruned
    ↓ embedding training
hotchpotch/bekko-embedding-v1-a8m
    ↓ language-selective vocabulary trimming: de,en,ru / 65,536 tokens
Winzling-Embed-a8m-64k FP32
    ↓ row-wise INT8 token embeddings
    ↓ MatMulNBits asymmetric UINT4 transformer weights
Winzling-Embed-a8m-64k UINT4 ONNX

No gradient-based training or fine-tuning is performed by the Winzling transformation itself.

Supported languages

This release is intentionally specialized for:

The tokenizer retains the required fallback machinery, so text in other scripts/languages may still tokenize and produce an embedding. That does not make those languages supported. Semantic quality outside de, en, and ru has not been validated and should not be assumed.

If broader multilingual coverage is required, use the original Bekko model or build a Winzling variant with the desired language set and a correspondingly larger vocabulary.

Quick start

ONNX / CPU

Install Sentence Transformers with its ONNX dependencies:

pip install "sentence-transformers[onnx]"

When the model repository is available locally, run from its root:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    ".",
    backend="onnx",
    model_kwargs={
        "file_name": "onnx/model_uint4.onnx",
        "provider": "CPUExecutionProvider",
    },
)

texts = [
    "Das ist ein deutscher Beispielsatz.",
    "This is an English example sentence.",
    "Это пример предложения на русском языке.",
]

embeddings = model.encode(
    texts,
    normalize_embeddings=True,
)

print(embeddings.shape)
# (3, 384)

similarities = model.similarity(embeddings, embeddings)
print(similarities)

For direct Hub loading, replace "." with this repository's full Hub model ID.

FP32 / PyTorch

The trimmed FP32 checkpoint remains useful for fine-tuning or for measuring quantization deltas:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(".")

embeddings = model.encode(
    [
        "Ein kurzer deutscher Text.",
        "A short English text.",
        "Короткий русский текст.",
    ],
    normalize_embeddings=True,
)

Using the Zstandard artifact

onnx/model_uint4.onnx.zst is a storage/distribution artifact. ONNX Runtime does not consume the .zst file directly.

Decompress it first:

zstd -d -k onnx/model_uint4.onnx.zst

This recreates:

onnx/model_uint4.onnx

while retaining the compressed file.

ONNX details

The ONNX model is intentionally mixed precision:

Token embedding table

The 65,536 × 384 token table is stored using row-wise INT8 quantization.

This table dominates the remaining model size even after vocabulary trimming:

65,536 × 384 = 25,165,824 embedding parameters

Transformer weights

Transformer matrix multiplications are quantized using ONNX Runtime's MatMulNBits representation:

• 4-bit unsigned weights
• asymmetric quantization
• block size 32
• weight-only quantization; activations are not UINT4

This is why calling the model simply "INT4 everywhere" would be inaccurate. It is a UINT4 transformer + INT8 token-table model.

Quantization preservation

During export, UINT4 output embeddings were compared with the 64k FP32 backbone:

Both gates passed.

The downstream MTEB regression provides a stronger task-level check: UINT4 reduces the compact-suite mean from 55.609 to 54.979, an incremental loss of 0.630 percentage points after vocabulary trimming.

Vocabulary trimming

The vocabulary was reduced from 256,000 to exactly 65,536 entries while preserving the transformer body.

The trimmer performs model-aware tokenizer surgery rather than simply slicing the first N token rows.

At a high level it:

1. selects the target languages (de, en, ru);
2. mines token usage from representative text for each language;
3. balances language contributions so one corpus does not dominate the vocabulary;
4. retains mandatory special/control tokens;
5. retains byte-fallback requirements;
6. recursively preserves BPE merge dependencies so retained composite tokens remain reachable;
7. builds a compact old-token-ID → new-token-ID mapping;
8. rewrites tokenizer vocabulary and merge tables;
9. remaps token-ID-bearing configuration fields;
10. slices the embedding matrix to the retained rows;
11. leaves the transformer encoder unchanged;
12. invalidates derived ONNX/OpenVINO artifacts and regenerates them from the trimmed checkpoint.

The exact corpus sources, revisions and transformation metadata should be recorded in the accompanying trim-manifest.json / trim-report.json generated by embedding-model-trimmer.

Why trimming can change embedding quality

Although the transformer itself is unchanged, vocabulary trimming changes how text is segmented.

A sentence that originally used a semantically useful learned token may be represented by a longer sequence of smaller pieces after trimming. This can alter the final embedding even before quantization.

The benchmark results below show that this effect is strongly task-dependent.

MTEB regression benchmark

Benchmark scope

This release was evaluated on a fixed 10-task regression suite derived from MTEB(Multilingual, v2) and filtered to German, English and Russian where available.

Scores shown here are:

100 × MTEB main_score

The suite is deliberately fixed so future dependency updates cannot silently change the benchmark composition.

It is not the official 131-task MMTEB Multilingual v2 aggregate.

The fixed task set is:

Overall

The incremental loss from quantizing the already-trimmed FP32 model is only:

54.979 - 55.609 = -0.630 percentage points

Most of the overall quality delta therefore comes from vocabulary trimming, not UINT4 quantization.

By task type

Per task

Interpreting the benchmark

The degradation is highly structured rather than uniform.

Tasks that remain close to the original Bekko model in this suite include:

• classification;
• pair classification;
• reranking;
• MIRACL hard-negative retrieval.

The largest losses appear in:

• cross-lingual / semantic textual similarity;
• BelebeleRetrieval;
• bitext mining.

This indicates that aggressive vocabulary specialization affects fine-grained multilingual semantic geometry much more than it affects every downstream capability equally.

The result should therefore not be interpreted as "a 64k vocabulary is lossless." It is not.

It should instead be interpreted as:

A large fraction of Bekko's model storage can be removed for a restricted language set while preserving surprisingly strong performance on several task families, but fine cross-lingual semantic alignment is materially degraded.

For applications dominated by STS, bitext mining or cross-lingual semantic retrieval, the original Bekko model or a larger-vocabulary trim is preferable.

For constrained-language classification, reranking, pair classification, downstream fine-tuning and edge deployment, this model occupies a substantially different size/quality point.

Upstream Bekko results

The upstream hotchpotch/bekko-embedding-v1-a8m model reports:

• 7.67M active parameters
• 105.98M total parameters
• 56.2 on official MMTEB Multilingual v2 retrieval
• 56.7 Mean(Task) over the official 131-task MMTEB Multilingual v2 suite
• approximately 124 MiB for its compact ONNX artifact

Those upstream scores use a different benchmark aggregate from the fixed de/en/ru suite above and must not be compared numerically as though they were the same benchmark.

Intended uses

Good candidate for:

• German/English/Russian semantic embeddings under tight storage constraints;
• edge and desktop applications;
• local semantic indexing;
• constrained-language retrieval where application-specific evaluation passes;
• reranking;
• pair classification;
• document/text classification;
• clustering after application-specific validation;
• feature extraction for small downstream heads;
• SetFit-style classification;
• language identification after training a dedicated classifier head;
• further fine-tuning from the trimmed FP32 checkpoint.

For downstream training, prefer the trimmed FP32 checkpoint and export/quantize only after fine-tuning.

Not intended for

This release should not be treated as:

• a drop-in replacement for the full multilingual Bekko model;
• a 100+ language embedding model;
• a state-of-the-art STS model;
• a lossless vocabulary-pruned checkpoint;
• a model with an official full-MMTEB score of 54.979;
• a guarantee of quality for languages other than German, English and Russian;
• a guarantee that UINT4 is faster than FP32 or INT8 on every runtime/hardware combination.

Compression ratio and inference speed are separate properties. MatMulNBits performance depends strongly on the ONNX Runtime execution provider and target CPU/GPU.

Fine-tuning

The FP32 trimmed checkpoint can be used as a normal Sentence Transformers backbone.

For classification-style tasks, a lightweight head can often be sufficient:

384-dimensional normalized embedding
        ↓
linear classifier
        ↓
task labels

For example, a three-language language-identification head requires only:

384 × 3 + 3 = 1,155 parameters

The quantized ONNX graph should generally be treated as a deployment artifact rather than the checkpoint from which to continue gradient-based training.

Matryoshka dimensions

The upstream Bekko model is trained to support truncated embedding dimensions:

• 384
• 256
• 128
• 64

This capability is inherited structurally because the transformer weights are preserved.

However, the lower dimensions have not been separately benchmarked for this vocabulary-trimmed release. Treat them as available but unvalidated until application-specific evaluation is performed.

Long-context behavior

The upstream encoder supports sequences up to 8,192 tokens.

That context limit remains in the model configuration, but vocabulary trimming changes tokenization density. The same source document may require more tokens after trimming.

Consequently:

• an 8,192-token model limit does not imply unchanged character/document capacity;
• long documents should be measured with the trimmed tokenizer;
• trim-report.json should be consulted for tokenization expansion statistics;
• truncation-sensitive applications should benchmark their actual corpus.

Reproducing the transformation

The core transformation is:

embedding-model-trimmer \
  --model hotchpotch/bekko-embedding-v1-a8m \
  --languages de,en,ru \
  --vocab-size 65536 \
  --output Winzling-Embed-a8m-64k \
  --export-onnx-uint4

For reproducible Hugging Face caching:

export HF_HOME=/path/to/huggingface-cache
export HF_TOKEN=hf_...

Then optionally create the distribution artifact:

zstd -19 -k Winzling-Embed-a8m-64k/onnx/model_uint4.onnx

The expected storage result for this release is:

onnx/model_uint4.onnx      29.1 MiB
onnx/model_uint4.onnx.zst  26.4 MiB

Exact reproducibility also depends on the corpus revisions and software versions recorded by the trimmer manifest.

Reproducibility and provenance

For scientific or production use, pin and retain:

• base model revision;
• tokenizer revision;
• target language list;
• target vocabulary size;
• token-frequency corpus revisions;
• old-ID → new-ID mapping;
• trimming manifest;
• trimming coverage report;
• ONNX Runtime version;
• INT4 block size and quantization mode;
• benchmark task revisions;
• model artifact checksum.

Do not identify a model only by a8m-64k: two 64k trims produced from different corpora or revisions can have different token vocabularies and therefore different behavior.

Files

A complete release should contain the normal Sentence Transformers / Transformers model files plus the transformation artifacts, including:

config.json
config_sentence_transformers.json
modules.json
sentence_bert_config.json
tokenizer.json
tokenizer_config.json
special_tokens_map.json
model.safetensors
1_Pooling/
onnx/
  model_uint4.onnx
  model_uint4.onnx.zst
trim-manifest.json
trim-report.json
README.md

model.safetensors is the trimmed trainable checkpoint.

onnx/model_uint4.onnx is the recommended compact runtime artifact.

onnx/model_uint4.onnx.zst is the smallest distribution/storage artifact and must be decompressed before inference.

Limitations

Language restriction

Only German, English and Russian are supported by this release.

Semantic-geometry degradation

Vocabulary trimming causes substantial losses on some STS, bitext and retrieval tasks. This is visible in the benchmark and should not be hidden behind the strong overall size reduction.

Quantization

UINT4 introduces a smaller additional quality loss. In the fixed benchmark, the aggregate incremental loss is 0.630 percentage points relative to the 64k FP32 model.

Backend performance

The smallest model is not automatically the fastest model.

Generic ONNX Runtime MatMulNBits execution can have very different performance characteristics across CPU and GPU providers. Benchmark the exact deployment environment.

Embeddings are not anonymization

Dense embeddings can retain information about their source text. Do not treat embedding generation as a privacy or anonymization mechanism.

License

This derivative is released under the MIT License, following the upstream hotchpotch/bekko-embedding-v1-a8m release.

The original Bekko architecture, pretrained/embedding-trained weights and research are the work of the Bekko authors. Winzling applies language-selective vocabulary surgery and deployment quantization to that model.

Citation

If you use this model, please cite the upstream Bekko work:

@misc{tateno2026bekko,
  title        = {Bekko Embedding: Parameter-Efficient Multilingual Retrieval with Ultra-Compact Encoders},
  author       = {Yuichi Tateno},
  year         = {2026},
  eprint       = {2607.25180},
  archivePrefix= {arXiv},
  primaryClass = {cs.CL}
}

When reporting results from this derivative, please also state the exact variant:

Winzling-Embed-a8m-64k
languages: de,en,ru
vocab: 65,536
ONNX: asymmetric UINT4 MatMulNBits / block size 32
token embeddings: row-wise INT8

and distinguish its fixed de/en/ru compact MTEB regression score from official full-MMTEB scores.

Acknowledgements

Winzling-Embed-a8m-64k exists because Bekko's architecture has an unusual and useful property for aggressive specialization: most of its total parameters live in the multilingual token embedding table, while the active encoder is only ~7.67M parameters.

That makes it possible to remove a large fraction of storage for a restricted language set without pruning transformer depth or width.

Thanks to Yuichi Tateno, Sentence Transformers, Hugging Face, MTEB and ONNX Runtime projects for the underlying models, tooling and evaluation infrastructure.