Nemotron-3.5-ASR-Streaming-Multilingual-0.6b-ov

Nemotron-3.5-ASR-Streaming-Multilingual 0.6B — OpenVINO IR

OpenVINO IR export of NVIDIA's nvidia/nemotron-3.5-asr-streaming-0.6b, a cache-aware streaming FastConformer-RNNT with prompt-conditioned multilingual decoding (40+ languages). This is the OpenVINO sibling of FluidInference/Nemotron-3.5-ASR-Streaming-Multilingual-0.6b-CoreML, intended for the eddy-audio runtime (OpenVINO, CPU / Intel NPU / iGPU).

Contents

fp32/   FP32 IR        (~2.5 GB)  — graph the WER/CER table below was measured on
fp16/   FP16 IR        (~1.3 GB)  — identical transcripts, half the size, NPU-friendly
int8/   INT8-weight IR (~749 MB)  — weight-only encoder (per-channel sym); WER matches FP32

Each folder holds 4 IR pairs plus the tokenizer + metadata:

Streaming / cache-aware I/O

• Chunk: chunk_mel_frames=112 (+pre_encode_cache=9 → total_mel_frames=121), att_context_size=[56,0].
• Encoder cache carried across chunks: cache_channel [1,24,56,1024], cache_time [1,24,1024,8], cache_len [1].
• Language is selected per chunk via an integer prompt_id (see prompt_dictionary in metadata.json; auto = 101).
• The 39 language-tag token ids in metadata.json["lang_tag_token_ids"] are stripped from the decoded text.

Accuracy (FLEURS, full test splits, FP32, greedy, forced language)

Measured on the OpenVINO IR (CPU). Reference column is the FluidAudio CoreML build of the same base model.

The export matches or beats the reference on every language — no conversion regression. FP16 produces transcripts identical to FP32.

Throughput: RTFx ≈ 3.66× audio-weighted on a single CPU (Intel Xeon E5-2699 v4, 4 vCPU); higher on Intel NPU / iGPU.

INT8 weight-only (int8/)

Weight-only INT8 compression of the encoder (per-channel symmetric, data-free — mirrors the CoreML linear_quantize_weights build). Only the weight constants become int8 with fp16 scales; activations stay fp16. The 24 Conformer relative-position projections (self_attn.linear_pos) are kept FP16 — int8-compressing them trips an OpenVINO CPU compile bug and they carry little weight mass. Decoder / joint / preprocessor stay FP16.

Measured on the same FLEURS full test splits, forced language, greedy — directly comparable to the FP32 table above:

Essentially lossless — the largest swing is +0.15 WER (within run-to-run noise) and Chinese CER improves slightly.

What INT8 buys you — and what it doesn't:

INT8 is a memory / disk win (~½ the RAM of FP16), not a speed win: on x86 CPU the int8 weights decompress to float per-op, so it runs slower than FP16. The speed payoff is on Intel NPU, where native int8 execution applies. Choose int8/ for memory-constrained or NPU deployments; choose fp16/ for fastest CPU inference.

License

Inherits openmdw-1.1 from the base model. See the base model card for terms.