nemotron-speech-streaming-en-0.6b-mlx-4bit

nemotron-speech-streaming-en-0.6b-mlx-4bit

4-bit quantized MLX conversion of nvidia/nemotron-speech-streaming-en-0.6b. See the original model card for training data, evaluation and intended use.

Variants: nemotron-speech-streaming-en-0.6b-mlx · nemotron-speech-streaming-en-0.6b-mlx-8bit · nemotron-speech-streaming-en-0.6b-mlx-4bit

Compatibility

Last checked 2026-10-14 by transcribing a sample recording in each runtime.

Usage

mlx-audio

# pip install -U mlx-audio
from mlx_audio.stt import load

model = load("animaslabs/nemotron-speech-streaming-en-0.6b-mlx-4bit")
print(model.generate("audio.wav").text)

mlx-voice-rs

cargo add mlx-voice
hf download animaslabs/nemotron-speech-streaming-en-0.6b-mlx-4bit --local-dir nemotron-speech-streaming-en-0.6b-mlx-4bit

use mlx_voice::asr::CacheAwareRNNTPipeline;
use mlx_voice::audio::load_wav;

fn main() -> mlx_voice::Result<()> {
    // Mode 0 is the lowest-latency streaming mode
    let mut pipeline = CacheAwareRNNTPipeline::restore_from_directory("nemotron-speech-streaming-en-0.6b-mlx-4bit", 0)?;
    let (samples, _sample_rate) = load_wav("audio.wav")?; // 16 kHz mono

    // Feed audio in 100 ms chunks, as it would arrive from a microphone
    let chunks: Vec<&[f32]> = samples.chunks(1600).collect();
    for (i, chunk) in chunks.iter().enumerate() {
        pipeline.add_audio(chunk);
        if i == chunks.len() - 1 {
            pipeline.end_stream();
        }
        while let Some(output) = pipeline.transcribe_step()? {
            if output.is_final {
                println!("{}", output.text);
            }
        }
    }
    Ok(())
}

Files

• model.safetensors: MLX-layout weights. Linear and embedding layers are 4-bit affine quantized with group size 64; the rest stays in floating point. LSTM weight matrices are float16.
• config.json: The model config in mlx-audio's nemotron_asr layout, plus a quantization block.
• model_config.yaml: The original NeMo config from the .nemo archive. mlx-voice-rs reads this one.
• tokenizer.model: SentencePiece tokenizer.
• tokenizer.vocab: SentencePiece vocabulary.
• vocab.txt: Token list.

Source

Converted from nvidia/nemotron-speech-streaming-en-0.6b at revision ebe59e5a81 (.nemo sha256 283638054c44…) using animaslabs/mlx-models.