wav2vec2-base-960h-gguf

wav2vec2-base-960h — GGUF

facebook/wav2vec2-base-960h converted to GGUF with F16 weights, for CTC forced alignment on ggml.

Not a transcription model in this form. It answers where a known word starts and ends: the decoder emits a per-frame character emission matrix, and a Viterbi pass over the CTC blank-expanded label sequence walks a word list onto the audio timeline. Stride is 320 at 16 kHz — one frame per 20 ms, which is the floor on timestamp precision.

Geometry: 32-token vocab, 768 hidden, 12 layers, 12 heads, 3072 FFN, group-norm CNN feature extractor (do_stable_layer_norm = 0).

Download

curl -L -O https://huggingface.co/janksmap/wav2vec2-base-960h-gguf/resolve/main/wav2vec2-base-960h-f16.gguf

The inference path these weights were built for

The reference wav2vec2.cpp implementation used ggml only as a one-shot matmul helper. Everything around it — the convolutional feature extractor, layer norms, GELU, the attention plumbing — was hand-written scalar and NEON loops, single-threaded, allocating a fresh ggml context per linear layer. The CNN front end alone is ~24.5 GMAC per 10 s of audio, and it ran on one core.

That forward pass was rewritten as a single ggml graph on ggml-backend, built once and executed through gallocr. Convolutions lower to im2col + mul_mat, so the whole encoder becomes one schedulable graph that the backend can thread and, where a backend is available, offload.

Numerical parity

Gated on word-span delta, not bit-exactness, and the reason matters. Both the old and new paths run ggml_mul_mat against F16 weights, and ggml-cpu rounds F32 activations to F16 before the dot product. A perturbation far below one F16 ulp still flips a fraction of activations by a full ulp, amplifying roughly 15× per matmul until it saturates at the F16 floor.

Control experiment: the unmodified original path, with its inputs nudged by a single float ULP, scores worse against golden logits (99.9010 % argmax, 20 ms span delta) than the rewrite does (99.9340 %, 0.00 ms). A bit-exactness gate would reject the original implementation, so it was retired in favour of a ≤ 20 ms word-span bound — one frame, the model's own resolution.

How it was written

The rewrite was authored with Claude Opus 5 orchestrating and Claude Sonnet 5 subagents doing the staged implementation and review — graph construction, weight loading, backend wiring, and an adversarial parity gate that disproved the original (unattainable) 100 %-argmax criterion by control experiment rather than accepting it.

License and attribution

Weights are a format conversion of facebook/wav2vec2-base-960h by Meta AI, Apache-2.0, which carries over here. Values are unchanged apart from F16 rounding — no fine-tuning, distillation, or retraining.

Original work: Baevski, Zhou, Mohamed, Auli, wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations, 2020 (arXiv:2006.11477).