Downloads · 30 days
10
7% of all-time downloads
Gimics/nemotron-3.5-asr-streaming-0.6b-onnx-int8
nemotron-3.5-asr-streaming-0.6b-onnx-int8 is a automatic speech recognition model from Gimics. Use it when you need speech turned into text. The card lists the license as other.
Weight-only dynamic int8 quantization of the ONNX export of NVIDIA's nemotron-3.5-asr-streaming-0.6b — a cache-aware streaming FastConformer + RNN-T multilingual ASR model (40 language-locales, with punctuation and ca…
Downloads · 30 days
10
7% of all-time downloads
All-time downloads
149
Public
Repo size
682 MB
Likes
0
Public
Click a slice to open those files.
.onnx682 MB · 100%
From the Hugging Face model README
Weight-only dynamic int8 quantization of the ONNX export of NVIDIA's
nemotron-3.5-asr-streaming-0.6b
— a cache-aware streaming FastConformer + RNN-T multilingual ASR model
(40 language-locales, with punctuation and casing).
This repackages the model for low-footprint local inference: ~2.45 GB → 651 MB, with transcription output verified identical to FP32 on English, German, and Russian samples.
| Stage | Source |
|---|---|
| Base model | nvidia/nemotron-3.5-asr-streaming-0.6b (OpenMDW-1.1) |
| ONNX export | altunenes/parakeet-rs — nemotron-3.5-asr-streaming-0.6b-onnx |
| int8 quantization | this repository |
Licensed under OpenMDW-1.1, inherited from the base model. All credit for the model and the ONNX export belongs to NVIDIA and the parakeet-rs author respectively; this repository only adds dynamic int8 quantization and a model card.
| File | Size | Notes |
|---|---|---|
encoder.int8.onnx | 658 MB | cache-aware streaming encoder (+ language prompt_index input) |
decoder_joint.int8.onnx | 24 MB | RNN-T decoder + joint network |
tokenizer.model | 0.4 MB | SentencePiece vocabulary (13,087 tokens) |
config.json | 3 KB | shapes + language → prompt-index dictionary |
The encoder is self-contained — no external .onnx.data sidecar.
from onnxruntime.quantization import quantize_dynamic, QuantType
quantize_dynamic("encoder.onnx", "encoder.int8.onnx", weight_type=QuantType.QInt8)
quantize_dynamic("decoder_joint.onnx", "decoder_joint.int8.onnx", weight_type=QuantType.QInt8)
Dynamic quantization is weight-only (MatMul / attention) and needs no calibration data. The subsampling Conv ops are left in FP32.
Decoded on CPU and compared against the FP32 export:
| Audio | Language | Output |
|---|---|---|
| JFK clip | en-US | "And so my fellow Americans ask not what your country can do for you. Ask what you can do for your country." |
| German | de-DE | "Am Strand der Badeanzug die Badehose, die Sandalen, die Luftmatratze, das Handtuch, …" |
| Russian | ru-RU | "Проверка связи." |
All outputs are identical to the FP32 model.
Load the encoder and decoder/joint ONNX graphs with an ONNX Runtime that
supports int8. The multilingual variant is identified by the encoder's
prompt_index input, which conditions decoding on a target language
(e.g. en-US, de-DE, or auto for automatic language detection).
40 language-locales across NVIDIA's transcription-ready, broad-coverage, and adaptation tiers. See the base model card for the full list and per-tier quality notes.