whisper-base

whisper-base

ONNX builds of openai/whisper-base.

Whisper is OpenAI's automatic speech recognition model, trained on 680 000 hours of multilingual audio. It transcribes and translates speech in 99 languages, and produces segment-level timestamps.

It reads audio in 30-second windows and decodes text one token at a time, which is what makes it robust to accents and background noise and also what makes it slower than models built for one language.

This checkpoint: 74 M parameters — the smallest that stays usable on hard audio.

Variants

Every variant is the same checkpoint at a different weight precision. Pick a row, download that folder, and read onnx_manifest.json in it for the graph names and their checksums.

fp32 is not published for this checkpoint — it is the source format, available from the original repository. It is used above as the reference the other rows are measured against.

Word error rate measured on ten minutes of English speech against a hand-checked transcript, greedy decoding, no voice-activity detection. It is a single corpus in a single language: use it to compare the rows against each other, not as an absolute score.

What the names mean

• fp16 — Half-precision encoder, full-precision decoder. Smaller than fp32 and slower on CPU, where no float16 kernel exists — useful on GPU.
• plain-int8 — 8-bit weights, nothing excluded.
• grain-int8-b32 — 8-bit weights, one scale per 32 values.
• edge-int8-1layer — 8-bit weights, with the outermost decoder layer in a different scheme.
• plain-int4 — 4-bit weights, nothing excluded. Usually the smallest that still works.
• sym-int4-asym — 4-bit weights with an asymmetric scale, which does not assume the weights are centred on zero.
• zone-int4-attn — 4-bit weights, with the attention projections kept at 8 bits.

The int4 variants are meant for the CPU. On ONNX Runtime's CUDA provider the 4-bit matrix kernel (MatMulNBits) loses precision above 128 rows, which covers every encoder pass.

Licence

These are format conversions. The weights, and the licence they come with, are those of the original checkpoint — see the source repository.