parakeet-tdt-0.6b-v3-MLX-Mixed-4bit8bit

Parakeet TDT 0.6B v3 — MLX Mixed 4-bit/8-bit

A multilingual mixed-precision MLX conversion of nvidia/parakeet-tdt-0.6b-v3, built from the verified unquantized FP32 mlx-community/parakeet-tdt-0.6b-v3 checkpoint.

The 0.401 GB safetensor uses group-64 affine Q4 for 217 encoder matrices and Q8 for 48 pointwise-convolution plus eight predictor/joint matrices. This is 17.96% smaller than the 0.489 GB sonic-speech/parakeet-tdt-0.6b-v3-int4 control while retaining its quality class on a complete six-language gate: 11.17% versus 11.21% macro WER. The 0.04-point difference is not presented as an accuracy improvement; the useful result is practical quality parity with less weight storage.

Complete multilingual evaluation

Measured locally on Apple Silicon with MLX and public parakeet-mlx inference. The gate scores every FLEURS test utterance for German, French, Dutch, Swedish, Danish, and Finnish: 4,509 utterances, 14.64 audio hours, and zero skipped references for every artifact. Macro WER gives each language equal weight.

Against FP32, this model removes 84.02% of weight storage for a 0.80-point macro-WER increase. RTFx includes audio loading, log-mel preprocessing, inference, and decoding, but excludes model loading and dataset preparation. Run order was sequential rather than thermally interleaved, so throughput is directional and should be remeasured on deployment hardware.

Allocation and rejected candidates

Calibration used two disjoint FLEURS validation clips per language (12 clips, seed 31415). Candidate selection used 25 held-out test rows per language (150 utterances, seed 42). Only the promoted candidate then ran over all test rows.

The accepted allocation is:

• Q4: encoder pre-projection and all attention/feed-forward matrices, 217 modules;
• Q8: every pointwise convolution plus predictor and joint matrices, 56 modules;
• BF16: ineligible normalization, residual, bias, and depthwise-convolution tensors.

The selection screen rejected uniform Q4 at 0.354 GB and 10.92% macro WER, and an English-v2-derived Q3/Q4 allocation at 0.367 GB and 11.12%. Isolated low-drift attention groups did not compose safely at Q3 across the multilingual model. Protecting convolution and decoder/head paths at Q8 produced the stronger size/quality point.

Provenance and reproducibility

• NVIDIA source revision: 7c35754d166cca382ad1e53e68b01e7c575f3a1d
• MLX Community FP32 revision: ed2b7e8c15f9aaa0b5772e2efb986255eaef7e15
• Sonic INT4 revision: aa25511e86a4a83285774ba03df07ca62069de2f

The Sonic artifact was audited directly. Its 217 packed matrices reproduce exactly by casting the pinned FP32 anchor to BF16 and applying MLX affine Q4/group-64; all remaining tensors equal the anchor cast to BF16.

The release was rebuilt from the pinned FP32 source in a clean output directory and reproduced the evaluated weights byte-for-byte:

model.safetensors SHA-256
8ca561e3d2425ebfc934c58c8c905c9717b020bb80f7c7b8abe6ded72e18872e

The exact resolved allocation is included as quantization_recipe.json. The compatibility loader is included as parakeet_unified_mlx.py; conversion code, sensitivity results, rejected profiles, and matched benchmark evidence live in yuting1214/model-lab.

Usage

Install the public runtime and loader dependencies:

pip install parakeet-mlx huggingface_hub pyyaml librosa soundfile

from huggingface_hub import snapshot_download
from parakeet_unified_mlx import load_parakeet_mlx

model_dir = snapshot_download(
    "MarkChen1214/parakeet-tdt-0.6b-v3-MLX-Mixed-4bit8bit"
)
model = load_parakeet_mlx(model_dir)
result = model.generate("speech.wav")
print(result.text)

Run the script from the downloaded repository directory, or place parakeet_unified_mlx.py on PYTHONPATH before importing it.

Scope and limitations

• The upstream model supports 25 European languages, but the local complete benchmark above covers only de_de, fr_fr, nl_nl, sv_se, da_dk, and fi_fi.
• de_de does not qualify Austrian or Swiss German, and fr_fr does not qualify Canadian or Quebec French. Regional product audio is still required.
• Measurements use offline full-context TDT inference and greedy decoding; this is not a native Swift or streaming-latency result.
• The included loader is required because current public parakeet-mlx does not instantiate per-module mixed bit widths from the saved recipe itself.
• Accuracy varies with accents, domains, noise, and recording conditions. Evaluate on the intended target distribution before deployment.

License

The upstream model is released under CC-BY-4.0. This quantized derivative keeps that license and attribution. See the upstream model card for details.