parakeet-tdt-0.6b-v2-mlx-6bit

Parakeet TDT 0.6b v2, MLX 6-bit

6-bit MLX quantization of nvidia/parakeet-tdt-0.6b-v2, built from mlx-community/parakeet-tdt-0.6b-v2 via mlx.nn.quantize(bits=6, group_size=64) (nn.Linear and nn.Embedding only; Conv1d, BatchNorm, and the TDT prediction LSTM stay bf16). English-only.

Benchmarks (LibriSpeech test-clean, 250 samples, M4): WER 1.87% (identical to bf16), RTF 0.022, peak MLX memory 1.13 GB.

Loading requires quantizing the module tree before loading weights; parakeet_mlx.from_pretrained alone will not load this checkpoint:

import json, mlx.nn as nn
from parakeet_mlx.utils import from_config

model = from_config(json.load(open("config.json")))
nn.quantize(model, group_size=64, bits=6)
model.load_weights("model.safetensors")

mel_filters.npy is the precomputed slaney mel filterbank (librosa.filters.mel(sr=16000, n_fft=512, n_mels=128, fmin=0, fmax=8000, norm="slaney")), shipped so inference environments can stub librosa out entirely.

Quantized for Spoke local dictation.