parakeet-tdt-0.6b-v3-onnx-q8

Parakeet TDT 0.6B v3 — weight-only int8 encoder (ONNX)

The encoder of ysdede/parakeet-tdt-0.6b-v3-onnx, requantized so that one file runs on both WebGPU and CPU in ONNX Runtime Web. Built by MeetScribe's scripts/quantize_parakeet_encoder.py; the decoder and vocabulary here are copied from the base repo unchanged.

The upstream int8 encoder is dynamically quantized — weights and activations both — which compiles to MatMulInteger and DynamicQuantizeLinear. Neither op has a WebGPU kernel, so that file is CPU-only, and quantizing activations costs real accuracy. This build keeps int8 weights and fp16 arithmetic, which compiles to MatMulNBits (block size 128) — implemented on WebGPU and on CPU.

Pointwise convolutions are rewritten as MatMul before quantizing, so the conformer conv modules quantize too instead of staying fp32.

Accuracy

Against the fp32 encoder over 47 s of English and Spanish speech: 1.9% encoder output error, and identical transcripts on every clip (0.0% WER). The upstream dynamic int8 encoder scores 44% encoder error on the same audio.

Use

Same interface as the base repo — parakeet.js or any ONNX Runtime Web setup. Inputs audio_signal [B, 128, T] (NeMo log-mel) and length; outputs outputs [B, 1024, T/8] and encoded_lengths. The identical weights are stored under both encoder-model.int8.onnx and encoder-model.fp16.onnx so a loader that picks a file per device gets this build either way.

In MeetScribe, point the frontend at this repo:

VITE_PARAKEET_MODEL_BASE=https://huggingface.co/<user>/<repo>/resolve/main/