parakeet-ultra-onnx-int8

parakeet-ultra-onnx-int8 — most accurate Parakeet for offline dictation

Offline transducer bundle of moondream/parakeet-ultra for sherpa-onnx (nemo_transducer, TDT). Drop-in replacement for parakeet-tdt-0.6b-v3-int8: same 4-file layout, same ~600 MB, measurably lower word error rate.

Download

from huggingface_hub import snapshot_download
snapshot_download("mldecode/parakeet-ultra-onnx-int8", local_dir="./parakeet-ultra-onnx-int8")

Or fetch individual files: https://huggingface.co/mldecode/parakeet-ultra-onnx-int8/resolve/main/<encoder.int8.onnx|decoder.int8.onnx|joiner.int8.onnx|tokens.txt>

Why this model

Parakeet Ultra keeps the full-precision parakeet-tdt-0.6b-v3 architecture (FastConformer 24×1024, TDT decoder, 25 European languages) and improves it with further training. This repo converts those weights to sherpa-onnx int8 so CPU apps (e.g. DictFlow) get the accuracy without a GPU stack.

Word error rate %, lower is better — upstream evaluations on identical pipelines (model card):

Biggest win over the ternary redux build: background noise (5.82 vs 9.04) — the case that matters for real-world dictation with room mics and office noise.

Speed / size (CPU)

Apple M2 CPU: sherpa int8 ≈ 28× realtime. (Single-thread greedy sanity check on a Windows dev machine: RTF ≈ 0.12, i.e. ~8× realtime; multi-threaded sherpa is much faster.) If you need maximum speed on CPU and dictate in quiet rooms, parakeet-redux + Photon is faster and smaller; if you need the best transcript on CPU, this is it.

Files

feat_dim=128, subsampling_factor=8, normalize_type=per_feature, vocab_size=8192 (+1 blank). Greedy and modified-beam-search TDT decoding supported.

Note: the encoder emits 640-dim frames (HF layout — projector folded into the encoder), whereas NVIDIA's NeMo export emits 1024-dim. The three files are self-consistent; do not mix halves with the v3 bundle.

Validation

• I/O contract matches offline-transducer-nemo-model.cc (positional inputs, int32 decoder tokens, [B,1,1,8198] joiner logits with duration head).
• sherpa_onnx.OfflineRecognizer(model_type="nemo_transducer") greedy decode of the reference EN sample: "Ask not what your country can do for you. Ask what you can do for your country." — exact match.
• ONNX↔PyTorch cosine similarity: decoder 0.997, joiner 0.999 (int8 noise only).

Use (Python)

import sherpa_onnx, soundfile as sf
rec = sherpa_onnx.OfflineRecognizer.from_transducer(
    encoder="encoder.int8.onnx", decoder="decoder.int8.onnx",
    joiner="joiner.int8.onnx", tokens="tokens.txt",
    model_type="nemo_transducer", feature_dim=128,
    decoding_method="greedy_search", num_threads=4)
audio, sr = sf.read("speech.wav", dtype="float32", always_2d=True)
s = rec.create_stream(); s.accept_waveform(sr, audio[:, 0]); rec.decode_stream(s)
print(s.result.text)

Provenance / license

• Source weights: moondream/parakeet-ultra (retrain of NVIDIA parakeet-tdt-0.6b-v3), exported from the HF transformers checkpoint with torch.onnx (opset 17) + ONNX Runtime dynamic quantization. Benchmarks above are upstream figures; int8 parity spot-checked (see Validation).
• License: CC-BY-4.0 — attribution to NVIDIA and Moondream.