Downloads · 30 days
68
25% of all-time downloads
GestaltLabs/parakeet-unified-en-0.6b-mixed-int4-int8
parakeet-unified-en-0.6b-mixed-int4-int8 is a automatic speech recognition model from GestaltLabs. Use it when you need speech turned into text. It is set up for nemo. The card lists the license as other.
Downloads · 30 days
68
25% of all-time downloads
All-time downloads
270
Public
Repo size
722 MB
Likes
0
Public
Click a slice to open those files.
.pt720 MB · 100%
From the Hugging Face model README
A weight-only post-training quantization (PTQ) of NVIDIA's
parakeet-unified-en-0.6b
English ASR model:
The encoder weights are stored in a custom packed tensor format
(packed_encoder/) plus the remaining FP32 state (non_encoder_tail.pt) and the
original NeMo runtime assets (config, tokenizer). A small pure-PyTorch script
(restore_nemo.py) verifies every payload file by SHA-256 and reconstructs a
standard .nemo checkpoint that loads with stock NeMo.
This is an unpack-to-float reference release: it proves the compressed weights survive a full-corpus accuracy evaluation, and gives you a compact artifact to store and ship. It is not a native low-bit inference engine — see Known issues.
This is a first exploration of compressing the Parakeet family, done as a measured baseline rather than a product: pick a simple, auditable quantization recipe, pack it bit-exactly, and score it on the full LibriSpeech dev-clean corpus against the untouched checkpoint. The goal was to measure the accuracy cost of one weight-only mixed-precision recipe and build reusable audit tooling: packed-state bit-exactness checks, fail-closed WER gates, and provenance records. Speech-to-speech and TTS are future interests, not capabilities demonstrated by this release. Nothing here is tested for transfer to those tasks.
Evaluated on LibriSpeech dev-clean (2,703 utterances, 40 speakers, 54,402 reference tokens) on a single NVIDIA A40, NeMo 3.0.0, torch 2.6.0+cu124, greedy decoding, CUDA graph decoder off. WER uses a punctuation-light normalizer — not comparable to official leaderboard numbers.
| Arm | Word errors | WER (%) ↓ | CER (%) ↓ | Restorable model payload |
|---|---|---|---|---|
| Original FP32 checkpoint | 934 / 54,402 | 1.7168 | 0.5090 | 2,474,055,680 B |
| This release (PTQ INT4/INT8) | 950 / 54,402 | 1.7463 | 0.5233 | 720,555,016 B |
| QAD pilot (rejected) | 951 / 54,402 | 1.7481 | 0.5216 | 720,555,016 B |

Notes on the numbers:
packed_audit.json, research/evidence/ptq-packed-audit.json).research/evidence/results.json (original + PTQ + QAD),
PROVENANCE.json (full run history, gates, and retractions).A quantization-aware distillation pilot — 16 AdamW updates, lr 1e-5, frozen teacher, normalized encoder-feature MSE, batch 1, on 128 training-split clips verified disjoint from the eval set — produced one more word error than plain PTQ (951 vs 950) and was rejected. This is a small bounded pilot, not a refutation of QAD/QAT generally; it only closes this specific recipe as a winning candidate. The published weights are the PTQ arm, not QAD output.
Step 1 — download this repo and restore a .nemo checkpoint (CPU, only
torch required; verifies SHA-256 of every payload file):
pip install "huggingface_hub[cli]" torch
hf download GestaltLabs/parakeet-unified-en-0.6b-mixed-int4-int8 \
--local-dir parakeet-mixed
cd parakeet-mixed
python restore_nemo.py --bundle . --output parakeet-mixed.nemo
Step 2 — transcribe in a separate CUDA environment with NeMo 3 and a compatible torch (the eval used torch 2.6.0+cu124):
pip install "nemo_toolkit[asr]>=3.0.0" "torch>=2.6" soundfile
import nemo.collections.asr as nemo_asr
from omegaconf import OmegaConf, open_dict
model = nemo_asr.models.ASRModel.restore_from("parakeet-mixed.nemo", map_location="cuda")
model.eval()
# Some checkpoints ship without validation_ds; patch it before decoding changes.
with open_dict(model.cfg):
if model.cfg.get("validation_ds") is None:
model.cfg.validation_ds = OmegaConf.create({"use_start_end_token": False})
# Keep the CUDA graph decoder off — that is the configuration all results
# above were measured with.
if model.cfg.get("decoding") is not None and model.cfg.decoding.get("greedy") is not None:
model.cfg.decoding.greedy.use_cuda_graph_decoder = False
if hasattr(model, "change_decoding_strategy") and model.cfg.get("decoding") is not None:
model.change_decoding_strategy(model.cfg.decoding, verbose=False)
# Audio must be 16 kHz mono.
hyps = model.transcribe(["clip_16k_mono.wav"], batch_size=4, num_workers=0,
verbose=False, use_lhotse=True)
print(hyps[0])
The restored checkpoint uses stock NeMo's floating-point execution path. It does not provide native low-bit acceleration or a demonstrated reduction in runtime VRAM. The fresh release converter is tested separately from the original A40 accuracy run; a fresh CUDA transcription of the release archive has not been run.
restore_nemo.py →
NeMo ASRModel.restore_from.PROVENANCE.json — complete run history: baselines, rejected arms, gates,
flagged artifacts, closed levers, and source-file hashes.research/evidence/ — evaluation JSONs and packed-state audits.research/src/parakeet_opt/ — the quantization and benchmarking code used.packed_audit.json — per-file SHA-256 manifest of this bundle.Prior work referenced: arXiv:2203.15952,
eschmidbauer/parakeet-unified-en-0.6b-c,
and the NVIDIA Model-Optimizer QAT documentation.
LibriSpeech attribution and dataset-license details are in EVIDENCE_NOTICE.
Canonical sanitized prediction files referenced by PROVENANCE.json are in
evidence/; research code and compact release summaries are in research/.
Weights are derived from nvidia/parakeet-unified-en-0.6b and inherit the
NVIDIA Open Model License — see LICENSE and NOTICE in this repo. This
is an independent release, not endorsed by NVIDIA.