NemotronLabs-VoiceChat-11B-mlx-8bit

NemotronLabs-VoiceChat-11B — MLX (8bit)

MLX conversion of nvidia/NVIDIA-NemotronLabs-VoiceChat-11B, a duplex speech-to-speech model that listens and speaks at the same time. 11.1 B parameters, converted from the original float32 release (44.4 GB) for Apple Silicon.

Recommended tier. Produces text identical to bf16 under greedy decoding while using 43 % less memory.

What is inside

The conversion keeps all four components of the model, complete and in the original layout.

Duplex operation runs at 80 ms frames (12.5 frames/s), 16 kHz input and 22.05 kHz output.

What runs in MLX today

Two parts of the model are directly usable, with runnable examples below:

• the language backbone, via mlx-lm's nemotron_h implementation
• the audio codec — waveform to residual-VQ codes and back

The Conformer speech encoder and the full duplex loop are not yet implemented in MLX. The weights for them are present and complete in this repository.

Example 1 — text generation with the language backbone

pip install mlx mlx-lm
python mlx/extract_llm.py --src . --dst ./voicechat-llm
python mlx/chat_example.py --model ./voicechat-llm --prompt "The capital of France is"

The source model ships no text tokenizer, so extract_llm.py pairs the backbone with a Nemotron-H base tokenizer (vocabulary 131072) and verifies the size matches before writing.

Example 2 — audio codec round trip

pip install mlx numpy
python mlx/codec_example.py --repo . --wav input.wav --out output.wav

Encodes a waveform to 512-dimensional latents and 31-stage codes, then decodes back to audio. Reconstruction is about 8 dB SNR at 12.5 frames per second, and runs roughly 6x faster than real time on an M4 Max.

Quantization

The language backbone and the three vocabulary heads are quantized to 8 bits (group size 64) — 9.47 B of the 11.10 B parameters. The entire speech path is left in bfloat16: the audio codec, the mixture-of-Gaussians head, the Conformer encoder and the RNN-T decoder. Those are small tensors whose error would land directly on audio quality for a negligible size saving.

Mean relative error against the bf16 tier, measured per tensor: 0.57 %.

Measured performance

Apple M4 Max, 68.7 GB unified memory, language backbone only, greedy decoding:

Audio codec: about 6x real time.

Notes

• Text-only prompting is outside the model's training distribution. It was trained to emit interleaved text and audio-codec tokens, so generations may end with audio-side tokens such as <SPECIAL_12>.
• The codec expects 22.05 kHz mono audio. Other rates decode at the wrong speed.
• Every tensor from the source checkpoint is present; the conversion is verified for completeness and for finite values in all tiers.

License and attribution

Released under OpenMDW-1.1, following the original model. Base model and architecture by NVIDIA; see nvidia/NVIDIA-NemotronLabs-VoiceChat-11B. The MLX audio-codec implementation follows NVIDIA NeMo Speech.