rekody-streaming-en-0.6b-int8

rekody-streaming-en-0.6b-int8

An int8 ONNX conversion of NVIDIA's Nemotron speech streaming 0.6B model (the March 2026 checkpoint), packaged for on-device streaming dictation. This is the model behind Rekody, the private, local-first voice dictation tool for macOS.

• Cache-aware streaming ASR: transcribes while you speak, in 560 ms chunks, with encoder cache carried across chunks. No audio leaves the machine.
• Runs on the ONNX Runtime CPU execution provider. The files load directly with onnxruntime or parakeet-rs.
• Four files: encoder.onnx (int8, 880 MB), decoder_joint.onnx (int8, 11 MB), tokenizer.model (SentencePiece), and config.json.

Why trust this conversion

• The conversion pipeline was validated bit for bit against the prior public int8 conversion of this architecture: when fed identical inputs (the same upstream checkpoint revision), our pipeline reproduces that artifact's tensors exactly. What ships here is the same pipeline applied to NVIDIA's current (March 2026) checkpoint.
• The cost of int8 quantization was measured as statistically zero. On LibriSpeech test-clean (n = 2620), int8 vs the fp32 export it came from differs by +0.072 pp WER with a paired bootstrap 95% CI of [-0.004, +0.150], which includes zero.

Benchmarks

Word error rate (WER, %), lower is better. Measured with the Open ASR Leaderboard methodology (hf-audio/open-asr-leaderboard @ b6bdcd0b, EnglishTextNormalizer), on identical audio and with the identical normalizer for both models.

Note the operating points: this model streams, producing text as audio arrives in 560 ms chunks, while whisper-large-v3-turbo is a batch model that sees each full recording before emitting anything. Matching a strong batch model's average WER under a streaming constraint is the point of this model.

License

Two licenses apply to this repository, and users must comply with both:

1. The conversion work (the int8 quantization, packaging, and benchmarking published here) is licensed under the PolyForm Shield License 1.0.0 (see LICENSE). PolyForm Shield is a noncompete license: any use is permitted except providing a product that competes with the software or with the licensor's products built on it.
2. The base model weights remain licensed by NVIDIA Corporation under the NVIDIA Open Model License (see NOTICE and the dated copy in licenses/).

File integrity (SHA-256)

These hashes are also published in SHA256SUMS.

Provenance

• Source: nvidia/nemotron-speech-streaming-en-0.6b, revision df1f0fe9dfdf05152936192b4c8c7653d53bf557 (the March 13, 2026 checkpoint).
• Conversion recipe: NeMo 2.7.3 ONNX export, then onnxruntime quantize_dynamic QInt8 over MatMul and LSTM ops.

Contact: [email protected]