parakeet-tdt-0.6b-v3-int8

sonic-speech/parakeet-tdt-0.6b-v3-int8

parakeet-tdt-0.6b-v3-int8 is a automatic speech recognition model from sonic-speech. Use it when you need speech turned into text. It is set up for mlx. The card lists the license as cc-by-4.0.

NVIDIA Parakeet TDT v3 with encoder-only INT8 quantization — the recommended variant for most users. Zero WER degradation, 30% faster, 58% less memory than BF16.

Downloads · 30 days

6% of all-time downloads

All-time downloads

1.6K

Public

Parameters

627M

755 MB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors755 MB · 100%

Parameter types

How the weights are stored.

U32533M · 85%

Base models

sonic-speech/parakeet-tdt-0.6b-v3

Model card

From the Hugging Face model README

Parakeet TDT 0.6B v3 (MLX, Encoder INT8)

NVIDIA Parakeet TDT v3 with encoder-only INT8 quantization — the recommended variant for most users. Zero WER degradation, 30% faster, 58% less memory than BF16.

Why INT8?

Metric	BF16	INT8	Change
WER (LibriSpeech)	0.82%	0.82%	None
WER (TED-LIUM)	15.1%	15.1%	None
RTFx	73x	95x	+30%
Peak Memory	3,002 MB	1,268 MB	-58%
Weight Size	1,254 MB	755 MB	-40%

Benchmarked on Apple M3 Max (64 GB), macOS Sequoia 15.7.3, MLX 0.30.4.

Quantization Details

Only the Conformer encoder (~85% of parameters) is quantized to INT8 (group_size=64). The decoder and joint network remain in BF16, preserving precision for token generation.

Usage

from parakeet import from_pretrained
import mlx.core as mx

model = from_pretrained("sonic-speech/parakeet-tdt-0.6b-v3-int8", dtype=mx.bfloat16)
result = model.transcribe("audio.wav")

Install: pip install parakeet-mlx

Origin

Quantized from sonic-speech/parakeet-tdt-0.6b-v3 using mlx.nn.quantize (encoder-only, 8-bit, group_size=64).

Part of the Sonic Speech model collection.

Task: Automatic Speech Recognition
Library: mlx
License: cc-by-4.0
Languages: en
Created: Mar 8, 2026
Updated: Mar 8, 2026