VibeVoice-ASR-BitNet-slim

VibeVoice-ASR-BitNet-slim

A repack of microsoft/VibeVoice-ASR-BitNet with a redundant tensor removed. No retraining; the ternary transformer body is byte-for-byte the original.

Runtime, kernels and speed engineering live in the VibeASR.cpp fork and are not repeated here. This card covers the weights: what changed, how accurate they are, and where they break.

What changed in the weights

The released LM GGUF stores output.weight as F16 (466.7 MB — 47% of the file) next to token_embd.weight as Q6_K. In the source checkpoint tie_word_embeddings is true and the two matrices are bit-identical, so the F16 tensor is the same matrix twice at higher precision. llama.cpp loads LLM_TENSOR_OUTPUT as optional and falls back to token_embd, so the duplicate is simply dropped: the output projection runs through the Q6_K copy. That is the only numerical change, and it also removes 466.7 MB from every decoded token's memory traffic.

Verified twice on independent suites: +0.30 and +0.32 corpus WER versus the original weights, with per-language deltas scattering in both directions — i.e. no measurable accuracy cost for a 27% size reduction.

Accuracy

Greedy decoding, corpus-level WER with digits and years spelled out on both sides. Two regimes, because they rank engines differently.

Short clips (5–25 s), 7 language/register sets

On short clips whisper is more accurate than this model, and whisper-small is also faster. If short-clip transcription accuracy is what you need, use whisper.

Per language (100-clip suite, this repo vs the original weights — the comparison this card is actually about):

(The absolute level differs from the table above because that one uses a 14-item two-regime suite; both compare engines measured in the same session.)

Long-form (real TED talks, whole recordings)

On genuine long-form this model is level with whisper-small — each wins one talk — and roughly 5 WER against a 2.4 on short clips, so the gap to whisper closes substantially as recordings get longer. whisper-turbo collapses into a repetition loop on both talks under whisper.cpp's default flags (its reference implementation has temperature-fallback logic that suppresses this).

Measured on real continuous speech (distil-whisper/tedlium-long-form), not on concatenated short clips — splicing independent utterances produces speaker jumps that break every engine and measure nothing.

Limits you must design around

• ~80 seconds per pass. Past that the decoder emits its end token early and silently drops the tail (measured: 101 s of audio → 119 of 280 reference words, with the token budget 98% unused). This is a property of the weights — the original checkpoint behaves identically. Chunk longer audio; 60 s chunks give the long-form numbers above.
• No speaker labels. The runtime can request a {Start, End, Speaker, Content} format, but that prompt targets the 7B checkpoint and this 1.5B model emits no speaker turns and transcribes ~1.4 WER worse when asked for it. The CLI rejects it. There is no diarization here.
• Language coverage. VibeVoice-ASR was trained on en, zh, fr, it, ko, pt, vi. Spanish and German are out of distribution but generalise usably; French is in-distribution yet weak on FLEURS' proper-noun-heavy register (~33 WER) and much better on read speech (~22 on MLS). Other languages degrade sharply. This repack does not change coverage.

Usage

Drop-in for the released model — same runtime, same flags:

./build/bin/asr_infer \
    --vae-model vibeasr-vae-encoder-i8_s.gguf \
    --lm-model  vibeasr-lm-i2_s-tied.gguf \
    --audio input.wav -t 4 --greedy

Domain terms can be biased at decode time with --hotwords a,b,c --hotword-boost 5 (token-trie logit boosting): measured on FLEURS-French with oracle terms it recovers 36.0 → 31.9 WER, and feeding the wrong clip's terms at the same strength does not degrade the baseline. λ=8 is past the stability knee.

Provenance

Produced with tools/requant_lm_head.cpp --drop. The VAE encoder and tokenizer files are copied unmodified from upstream. Licensed MIT, as upstream.