parakeet-tdt-0.6b-ja-GGUF

Parakeet TDT-CTC 0.6B (Japanese) — GGUF

GGUF / ggml conversions of nvidia/parakeet-tdt_ctc-0.6b-ja for use with the crispasr CLI from CrispStrobe/CrispASR.

A 600 M-parameter Japanese ASR model with punctuation:

• Hybrid FastConformer-TDT-CTC: TDT (Token-and-Duration Transducer) decoder by default; the CTC head is included in these GGUFs and selectable at runtime with --parakeet-decoder ctc.
• Built-in word-level timestamps from the TDT duration head — no separate CTC alignment.
• 6.4 % CER on JSUT basic5000.
• CC-BY-4.0 licence.

Files

All files include both the TDT decoder and the CTC head (added 2026-07; earlier uploads lacked the CTC tensors, so --parakeet-decoder ctc silently fell back to TDT).

Recommended: F16

Verified on a JSUT-basic5000 sample at F16:

NeMo (PyTorch): '水をマレーシアから買わなくてはならないのです。'
crispasr (F16): '水をマレーシアから買わなくてはならないのです。'

The F16 GGUF produces an identical transcript to the official NeMo Python pipeline.

About the Q4_K variant

The Japanese model uses an 80-mel preprocessor (vs. 128 for the multilingual v3) and a smaller, more sensitive encoder distribution. With our default Q4_K quantisation, two of the most logit-shaping tensors (joint.pred.weight, decoder.embed.weight) fall back to q4_0 because their dimensions don't tile cleanly for q4_k blocks. This is enough quantisation noise that the TDT decoder enters a fixed-point loop after the first ~8 tokens. Two clean options: prefer the Q8_0 (TDT output identical to F16 in our tests), or keep the Q4_K and decode with the CTC head (--parakeet-decoder ctc) — CTC has no autoregressive feedback, so the quantisation noise doesn't compound, and its output matches the F16 CTC transcript.

Quick start

# 1. Build the runtime
git clone https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc) --target crispasr-lib

# 2. Download the F16
huggingface-cli download cstr/parakeet-tdt-0.6b-ja-GGUF \
    parakeet-tdt-0.6b-ja.gguf --local-dir .

# 3. Transcribe a 16 kHz mono WAV
./build/bin/crispasr --backend parakeet \
    -m parakeet-tdt-0.6b-ja.gguf -f your-japanese-audio.wav -t 8

You can also let crispasr auto-download the model:

./build/bin/crispasr --backend parakeet -m auto --auto-download \
    -f your-japanese-audio.wav --model-name parakeet-ja

Long-form audio (v0.8.8+)

The encoder is numerically fragile past ~12 s of context on real speech — single-pass decoding of long audio silently drops content (upstream NeMo behaves the same on the same clips: its plain, local-attention, and buffered long-form modes score 1–51 % content recall on our reference clip). CrispASR ≥ 3a8141e3 handles this automatically: audio > 30 s is VAD-segmented, slices are capped at 12 s (split at energy minima), each slice decodes in one NeMo-exact pass, and a gap-fill second pass re-transcribes any span the first pass left empty. Measured on the issue #89 reporter's clips (phonetic char-bigram recall vs whisper-large-v3-turbo): 97.2 % (60 s), 96.9 % (120 s), 95.9 % (300 s) — at the inter-model agreement ceiling (an independent SenseVoice-small run scores the same recall on the same audio). No flags needed; --vad, --chunk-seconds N, and CRISPASR_PARAKEET_* env vars override the defaults (see the CrispASR CLI docs).

Word-level timestamps for free

Pass -v to dump per-token timestamps from the TDT duration head. Each token spans one or more encoder frames; one frame = 80 ms. No separate alignment model required.

Model architecture

xscaling=True is the most important architectural detail vs. the multilingual v3 model — nvidia/parakeet-tdt-0.6b-v3 uses xscaling=False. Both settings are stored in the GGUF metadata (parakeet.xscaling) and read by the runtime, so the same code path serves both variants without per-model branches.

How this was made

1. The .nemo checkpoint is unpacked; every architecture hyperparameter (d_model, n_layers, ff_dim, pred_hidden, joint_hidden, xscaling, …) is read from model_config.yaml and cross-checked against the actual tensor shapes. The mel filterbank and Hann window are baked directly into the GGUF (preprocessor.fb, preprocessor.window).
2. NeMo state-dict keys are remapped to ggml-friendly names. Weights are written as F16 for matmul tensors and F32 for norms / biases / mel filterbank. A synthetic zero conv.dw.bias is added per encoder layer when the checkpoint omits it (older NeMo BN-only convs).
3. Inference is implemented in src/parakeet.{h,cpp}: the FastConformer encoder runs as a single ggml graph (BN folded into the depthwise conv weights at load time, xscaling applied between the pre-encode and the first block when parakeet.xscaling=true), the LSTM predictor and joint head run as manual F32 CPU loops, and the TDT greedy decode loop alternates "advance encoder frame" / "emit token + advance predictor" using the duration head's argmax.

Attribution

• Original model: nvidia/parakeet-tdt_ctc-0.6b-ja (CC-BY-4.0). NVIDIA NeMo team.
• GGUF conversion + ggml runtime: CrispStrobe/CrispASR.

Related

• Multilingual sibling (25 EU languages): cstr/parakeet-tdt-0.6b-v3-GGUF
• C++ runtime: CrispStrobe/CrispASR

License

CC-BY-4.0, inherited from the base model. Use of these GGUF files must comply with the CC-BY-4.0 license including attribution.

Provenance and EU AI Act Art. 53 note

• Upstream model: nvidia/parakeet-tdt_ctc-0.6b-ja — published by nvidia.
• Upstream licence: cc-by-4.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.