Downloads · 30 days
18
12% of all-time downloads
littlebearlabs/parakeet-unified-en-0.6b-mlx-int4
parakeet-unified-en-0.6b-mlx-int4 is a automatic speech recognition model from littlebearlabs. Use it when you need speech turned into text. It is set up for mlx. The card lists the license as cc-by-4.0.
4-bit affine-quantized MLX weights for NVIDIA's parakeet-unified-en-0.6b Cache-Aware FastConformer-RNNT, for the witness MLX C++ engine on Apple Silicon.
Downloads · 30 days
18
12% of all-time downloads
All-time downloads
151
Public
Parameters
613M
391 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors391 MB · 100%
How the weights are stored.
U32611M · 100%
From the Hugging Face model README
4-bit affine-quantized MLX weights for NVIDIA's
parakeet-unified-en-0.6b
Cache-Aware FastConformer-RNNT, for the witness MLX C++ engine on Apple Silicon.
Same quantization scope as the int8 variant (linear/projection matmuls only, group size 64; conv2d / depthwise / norms / biases / position bias stay dense).
int4 is the smallest variant (0.39 GB) and is WER-neutral for the offline
(full-context) path, so it is a good fit for memory-constrained offline
transcription. For the streaming (cache-aware chunked) path it shows a small
WER regression vs int8/dense — prefer
-int8
for streaming. The int4 RTF is within noise of int8, so the trade is memory, not
speed.
| Variant | Size | Offline WER | Offline RTF | Streaming WER | Streaming RTF |
|---|---|---|---|---|---|
| dense fp32 | 2.47 GB | 1.78% | 0.0084 | 11.35% | 0.0319 |
| int8 | 0.70 GB | 1.78% | 0.0075 | 11.35% | 0.0197 |
| int4 | 0.39 GB | 1.78% | ~0.0080 | 12.68% | 0.0206 |
WITNESS_PARAKEET_UNIFIED_MODEL_DIR=/path/to/this/dir
Produced by crates/mlx-parakeet/scripts/quantize_parakeet_unified.py --bits 4.