parakeet-cpp-gguf-fork

Parakeet GGUF — models for parakeet.cpp

GGUF-format weights for parakeet.cpp, a C++/ggml port of NVIDIA NeMo Parakeet that matches the upstream PyTorch models on CPU. This single repo collects every supported model × quantization as a flat set of .gguf files — download just the one you need.

F16 is the recommended default — same accuracy as F32, ~1.7× smaller, and typically the fastest on modern CPUs via ggml's F32×F16 matmul fast path.

Models

tdt_ctc-110m

Source: nvidia/parakeet-tdt_ctc-110m · Hybrid TDT+CTC (FastConformer) · heads: TDT + CTC

realtime_eou_120m-v1

Source: nvidia/parakeet_realtime_eou_120m-v1 · Cache-aware streaming RNNT (FastConformer, EOU/EOB) · heads: RNNT (streaming)

ctc-0.6b

Source: nvidia/parakeet-ctc-0.6b · CTC (FastConformer) · heads: CTC

rnnt-0.6b

Source: nvidia/parakeet-rnnt-0.6b · RNNT transducer (FastConformer) · heads: RNNT

tdt-0.6b-v2

Source: nvidia/parakeet-tdt-0.6b-v2 · TDT transducer (FastConformer) · heads: TDT

tdt-0.6b-v3

Source: nvidia/parakeet-tdt-0.6b-v3 · TDT transducer (FastConformer) · heads: TDT

ctc-1.1b

Source: nvidia/parakeet-ctc-1.1b · CTC (FastConformer) · heads: CTC

rnnt-1.1b

Source: nvidia/parakeet-rnnt-1.1b · RNNT transducer (FastConformer) · heads: RNNT

tdt-1.1b

Source: nvidia/parakeet-tdt-1.1b · TDT transducer (FastConformer) · heads: TDT

tdt_ctc-1.1b

Source: nvidia/parakeet-tdt_ctc-1.1b · Hybrid TDT+CTC (FastConformer) · heads: TDT + CTC

WER (word error rate) is computed against the upstream NeMo reference on tests/fixtures/speech.wav (LibriSpeech 2086-149220-0033, ~7.4 s, English). 0.0 = byte-for-byte identical transcript. See parity.md and quantization.md.

Quantization notes

Quantization is applied only to the large linear weights fed directly into ggml_mul_mat (encoder FFN + attention projections, subsampling output projection, joint enc/pred projections). All other tensors (mel filterbank, LSTM prediction net, conv kernels, batch_norm stats, norms, biases, embeddings) stay F32.

Usage

# 1. Clone + build parakeet.cpp
git clone https://github.com/mudler/parakeet.cpp
cd parakeet.cpp
cmake -B build -DPARAKEET_BUILD_CLI=ON && cmake --build build -j

# 2. Download one quant (F16 recommended)
huggingface-cli download mudler/parakeet-cpp-gguf tdt_ctc-110m-f16.gguf --local-dir models/

# 3. Transcribe
build/examples/cli/parakeet-cli transcribe \
    --model models/tdt_ctc-110m-f16.gguf \
    --input audio.wav

License

The GGUF weights are derived from the NVIDIA NeMo Parakeet checkpoints, released under the CC-BY-4.0 license. The parakeet.cpp runtime is MIT-licensed.