moss-transcribe-preview-2b-gguf

MOSS-Transcribe-preview-2B: calibrated GGUF (Starling engine)

A block-quantized GGUF of OpenMOSS-Team/MOSS-Transcribe-preview-2B (Apache-2.0), built with Starling's in-tree quantization pipeline. The linear layers use importance-matrix-weighted Q4_0, and the tied embedding / lm_head uses Q8_0.

Runtime note: this file follows the Starling GGUF tensor contract and runs on the native starling-serve binary / libstarling_ggml engine from the starling repository, including its Vulkan fast engine. It is not a llama.cpp / whisper.cpp GGUF.

File

SHA-256: 5658f3107a72bc7d74c3c428615fde9c95b439a436b9a3b1f387cf2f82a439ff

Recipe: quants/recipes/moss-q4e8-fullimx.recipe in the starling repository (starling-quantize --recipe … --imatrix … --f32-1d from the BF16-exact conversion).

moss-transcribe-preview-2b-q4e4-cal.gguf (phone default)

Q4_0 linears and a Q4_0 tied head with an importance matrix collected on 160 FLEURS validation clips (moss-calib160.imx, which covers the head; the older matrix did not). Recipe: moss-q4e4-cal.recipe (--f32-1d). FLEURS en_us test, 100 clips, vs BF16 (8.11 %): 8.02 % (paired Δ −0.18 [−1.11, +0.75]); q4e8 7.59 %. MOSS mostly translates German speech, so German is not reported. Pixel 10 Pro fast engine vs q4e8: load 3.4 vs 4.5 s, 22 s clip 14.3 vs 14.8 s, peak RSS 3.05 vs 3.51 GB.

SHA-256: 380b4e1d34de22511196d62de20da52c2b698e046f062b5eee7db810564d4066

Measured quality

FLEURS en_us test, first 100 clips, corpus WER:

Speed (Starling fast engine)

Usage

hf download scholzmx/moss-transcribe-preview-2b-gguf \
  moss-transcribe-preview-2b-q4e8-fullimx.gguf --local-dir ./models
starling-serve --model moss \
  --gguf ./models/moss-transcribe-preview-2b-q4e8-fullimx.gguf --port 8181

License

Apache-2.0, inherited from the base model. Quantization by the Starling project.