granite-embedding-311m-multilingual-r2-F16-GGUF

granite-embedding-311m-multilingual-r2 — F16 GGUF (provenance-documented conversion)

An F16 GGUF conversion of ibm-granite/granite-embedding-311m-multilingual-r2 for serving with llama-server, published with a complete, verifiable provenance chain. IBM publishes no GGUF of this model; existing third-party conversions have unverifiable provenance — this repo documents its build at checksum level so you can verify rather than trust.

Provenance

Conversion command:

docker run --rm -v "$PWD/src:/src:ro" -v "$PWD/out:/out" \
  ghcr.io/ggml-org/llama.cpp:full-cuda13@sha256:500f241b5d7e27846bdda69ce6f4889db8de859ea3988d4265e5d9d7b7f1079b \
  --convert /src --outtype f16 --outfile /out/granite-embedding-311m-multilingual-r2-F16.gguf

Fidelity check

Verified against the official safetensors pipeline (sentence-transformers): cosine(reference, served) over a 66-probe multilingual set (Romanian, three diacritic-variant forms) — min cosine 0.99993.

Serving

llama-server -m granite-embedding-311m-multilingual-r2-F16.gguf \
  --embeddings --pooling cls -ngl 999 --ctx-size 2048

• --pooling cls must be explicit — NOT mean. The model's own 1_Pooling/config.json says pooling_mode_cls_token=true; mean pooling silently degrades every embedding. The GGUF carries the correct pooling type in its metadata, but the explicit flag keeps the contract auditable.
• 768-dim output, plain queries and passages (no instruction protocol), 32k max context, ModernBERT architecture, ~0.7 GiB at F16 — no reason to quantize at this size.

License & attribution

Apache-2.0, same as the base model. All model weights are © IBM (ibm-granite/granite-embedding-311m-multilingual-r2); this repo contributes only the format conversion and its provenance documentation.