MOSS-Transcribe-preview-2B-GGUF

MOSS-Transcribe-preview-2B — GGUF (ggml-quantised)

GGUF / ggml conversions of OpenMOSS-Team/MOSS-Transcribe-preview-2B for use with the crispasr CLI from CrispStrobe/CrispASR.

MOSS-Transcribe is OpenMOSS's speech-LLM ASR model (~2.4 B params, Apache-2.0):

• Stock Qwen3-Omni-MoE audio encoder (the full 1280-dim / 32-layer tower) feeds frames into a Qwen3-1.7B LLM via embedding splice in a ChatML prompt.
• 4.87 % average WER (reported by the authors).
• Runs on CPU or GPU (Metal/CUDA) through the CrispASR runtime, with a persistent KV cache for O(1) per-token decode.

It is a close sibling of CrispASR's moss-audio backend (same author) but ASR-dedicated: no DeepStack, a conv_out/proj1/proj2 encoder head, and a smaller 1.7 B decoder.

Files

The Q4_K and Q8_0 builds keep the audio encoder, the adapter, and the tied token-embedding / output head at F16 (only the LM's attention and FFN matmuls are quantised), so transcript quality is preserved.

All quantisations produce the correct transcript on samples/jfk.wav:

and so my fellow americans ask not what your country can do for you ask what you can do for your country

(The model outputs lowercase, lightly punctuated text.)

Quick Start

# 1. Build the runtime
git clone https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc) --target crispasr

# 2. Download a quantisation
hf download cstr/MOSS-Transcribe-preview-2B-GGUF \
    moss-transcribe-preview-2b-q4_k.gguf --local-dir .

# 3. Transcribe
./build/bin/crispasr -m moss-transcribe-preview-2b-q4_k.gguf your-audio.wav

Audio must be 16 kHz mono. Pre-convert with:

ffmpeg -i input.mp3 -ar 16000 -ac 1 -c:a pcm_s16le output.wav

The CrispASR model registry also auto-downloads the Q4_K build on demand (-m moss-transcribe).

Architecture

Implementation notes (correctness)

The C++ runtime is verified against the PyTorch reference at every architectural boundary on samples/jfk.wav via the crispasr-diff harness:

Non-obvious gotchas the port handled

1. Prompt template is mandatory. Inference must use the chat_template_default.py ChatML framing (user / assistant markers around the audio). The bare audio layout makes the model emit garbage instead of transcribing.
2. Whisper drops the trailing STFT frame (stft[..., :-1]), giving exactly n_samples / hop mel frames; the runtime truncates to match (otherwise the audio-token count drifts by one).
3. The token embedding is tied to the output head, so it is pinned at F16 in the quantised builds — quantising it corrupts both the input embeddings and every output logit.

License

Apache-2.0, inherited from the base model.

Provenance and EU AI Act Art. 53 note

• Upstream model: OpenMOSS-Team/MOSS-Transcribe-preview-2B — published by OpenMOSS-Team.
• Upstream licence: apache-2.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.