Downloads · 30 days
46
44% of all-time downloads
dys-asr/parakeet-rnnt-0.6b-all-syn-chunk-cutout-bpedrop
parakeet-rnnt-0.6b-all-syn-chunk-cutout-bpedrop is a automatic speech recognition model from dys-asr. Use it when you need speech turned into text. It is set up for transformers. The card lists the license as other.
The ten-epoch recipe behind dys-asr/parakeet-rnnt-0.6b-all-syn-chunk-cutout, rerun with BPE-dropout at p = 0.1 on the training targets and nothing else changed. Each merge is skipped with that probability when a trans…
Downloads · 30 days
46
44% of all-time downloads
All-time downloads
104
Public
Parameters
618M
2.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.5 GB · 100%
From the Hugging Face model README
The ten-epoch recipe behind
dys-asr/parakeet-rnnt-0.6b-all-syn-chunk-cutout,
rerun with BPE-dropout at p = 0.1 on the training targets and nothing else
changed. Each merge is skipped with that probability when a transcript is
tokenised, so the same sentence is segmented differently from epoch to epoch and
the prediction network never settles on one spelling of a word.
On the held-out dev split, against the run it varies:
| run | best epoch | dev CER (4,000-utterance subset) | dev CER (full split) |
|---|---|---|---|
| baseline, no change | 9 | 6.06% | 6.25% |
| this model | 10 | 6.04% | 6.09% |
Treat both columns with suspicion. The subset and the full split disagree about the ordering across this family of runs, the spread is under 0.2 CER points, and there is one seed behind each row. The full split is 17,492 utterances against the subset's 4,000 and is the more trustworthy of the two, but neither is a significance test.
Every run in this family peaked at epoch 9 or 10 with no sign of overfitting, so ten epochs may simply be short.
ParakeetProcessor.__call__ tokenises the transcript twice — once for
labels, and again for decoder_input_ids after prepending the blank token.
With sampling on, those two calls draw different segmentations of the same
sentence, the joint's U+1 stops matching max(target_lengths) + 1, and
torchaudio raises output length mismatch on the first step. The training
collator here tokenises once and builds the decoder input from that single
sample. With dropout off the result is bit-identical to what the processor
produces, which is what keeps the other arms comparable.
Sampling also lengthens targets: an 18-token sentence measured between 18 and 22 tokens on this tokeniser. The evaluation tokeniser is a separate object and is never given a dropout, so dev scores do not move with the sampling.
The same corpus as
dys-asr/parakeet-rnnt-0.6b-all-syn-chunk-cutout,
unchanged, so the two are directly comparable.
| source | records |
|---|---|
| SAPC-1 train and dev | 250,014 |
| SAPC-2 train | 153,500 |
| synthetic CosyVoice3 speech | 55,988 |
| chunks force-aligned out of over-length recordings | 8,933 |
| total before filtering | 468,435 |
463,177 records survive the 0.5-45 s and 200-label-token filters. Nothing comes from outside the challenge corpora, so this is a constrained-track model.
The held-out dev split is genuinely held out. sapc2_dev_heldout.jsonl is
17,582 clips and its intersection with the training manifests is empty; that was
checked by comparing file paths, not assumed.
Ten epochs on sixteen GH200s, effective batch 32 at per-device 2, AdamW at 1e-4 with a tri-stage schedule (10% warmup, 40% hold), weight decay 0.01, layerdrop 0.05, gradient clip 1.0, bf16, seed 42, 144,750 optimizer steps. Augmentation is online speed perturbation over 0.8-1.2, SpecAugment (5% of the time axis in spans of 10 frames, 40% of the mel axis in spans of 27 bins) and SpecCutout (two 20x20 rectangles).
import soundfile as sf
import torch
from transformers import AutoProcessor, ParakeetForRNNT
model_id = "dys-asr/parakeet-rnnt-0.6b-all-syn-chunk-cutout-bpedrop"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained(model_id)
model = ParakeetForRNNT.from_pretrained(model_id).to(device).eval()
audio, rate = sf.read("utterance.wav", dtype="float32")
inputs = processor(audio, sampling_rate=rate, return_tensors="pt").to(device)
with torch.inference_mode():
generated = model.generate(**inputs)
print(processor.batch_decode(generated, skip_special_tokens=True)[0])
Requires transformers>=5.9. Audio must be 16 kHz mono.
A competition entry for the Speech Accessibility Project Challenge, and research on atypical-speech recognition.
transformers>=5.9. Not loadable by the 4.x line.The Speech Accessibility Project corpora are governed by their own data use agreement and are not redistributed here; reproducing this training set requires authorized access. Fun-CosyVoice3's terms apply to the synthesis component.