parakeet-ctc-0.6b-legal2023

parakeet-ctc-0.6b-legal2023

A fine-tune of NVIDIA's nvidia/parakeet-ctc-0.6b (FastConformer-CTC, ~0.6 B params, English) adapted to Indian legal / court-proceeding audio. On the held-out test set it reaches 18.27 % WER (greedy, no language model) versus 23.39 % for the base model — a 5.1-point (≈22 % relative) improvement.

• Base model: nvidia/parakeet-ctc-0.6b (BPE tokenizer kept unchanged)
• Training data: vanarp/legal2023_38hrs (train split, 35.66 h)
• Output style: lowercase, no punctuation
• Decoding used for the headline number: greedy, no language model

⚠️ The transcripts are pseudo-labels (pipeline-derived, not human gold) and the audio is restricted court-proceeding data — read Limitations and License below before use.

---

How to use — reproduce the test WER

The test split of vanarp/legal2023_38hrs is already silence-padded and matches the reported number. The snippet below downloads the model, transcribes the test split greedily, and computes corpus WER with the same normalizer used to produce 18.27 % (lowercase, keep [a-z0-9 ], collapse whitespace; word-level Levenshtein; total edits ÷ total reference words).

pip install "nemo_toolkit[asr]==2.7.3" "datasets>=2.18" huggingface_hub
# torch 2.8 (cu128) is pulled in by NeMo; a GPU is recommended.

import re
from datasets import load_dataset
from huggingface_hub import hf_hub_download
from nemo.collections.asr.models import ASRModel

# 1. model + test split  (set REPO_ID to where you upload this model)
REPO_ID = "vanarp/parakeet-ctc-0.6b-legal2023"
ckpt = hf_hub_download(REPO_ID, "parakeet-ctc-0.6b-legal2023.nemo")
model = ASRModel.restore_from(ckpt)                       # tokenizer is bundled in the .nemo

ds = load_dataset("vanarp/legal2023_38hrs", split="test")  # 2258 padded segments, 26 speakers

# 2. transcribe (greedy) — audio is stored inline as 16 kHz float arrays
audio = [row["array"].astype("float32") for row in ds["audio"]]
hyps = model.transcribe(audio, batch_size=32)
hyps = [h.text if hasattr(h, "text") else h for h in hyps]  # NeMo may return Hypothesis objects

# 3. normalize exactly like the original pipeline
_PUNCT, _WS = re.compile(r"[^a-z0-9 ]+"), re.compile(r"\s+")
def norm(t): return _WS.sub(" ", _PUNCT.sub(" ", t.lower())).strip()

# 4. corpus WER = total word-level edits / total reference words
def word_edits(r, h):
    dp = list(range(len(h) + 1))
    for i, rw in enumerate(r, 1):
        prev, dp[0] = dp[0], i
        for j, hw in enumerate(h, 1):
            prev, dp[j] = dp[j], min(dp[j] + 1, dp[j - 1] + 1, prev + (rw != hw))
    return dp[-1]

edits = words = 0
for ref, hyp in zip(ds["text"], hyps):
    r, h = norm(ref).split(), norm(hyp).split()
    edits += word_edits(r, h); words += len(r)

print(f"corpus WER = {100 * edits / words:.2f}%")   # -> ~18.27%

To transcribe your own audio, pass 16 kHz mono WAV paths (or float arrays) to model.transcribe([...]).

---

Results — legal2023_38hrs test (2258 segments, padded, greedy, no LM)

The fine-tune also beats the independent rnnt-1.1b judge (20.03 %), which is the honest, non-circular yardstick (see Limitations). 25 of 26 test speakers improved.

Number recognition (spoken cardinals/ordinals in the references): mis-recognized numbers (substitution + deletion) dropped 17.78 % → 15.10 %, and hallucinated (inserted) numbers dropped 114 → 50.

---

Training

• Init: nvidia/parakeet-ctc-0.6b; pretrained BPE tokenizer kept (no vocab rebuild).
• Learning rate: AdamW, peak 1e-4 (10–50× below from-scratch), betas [0.9, 0.98], weight-decay 1e-3, grad-clip 1.0. Cosine-annealing with 1000-step linear warm-up → min-lr 1e-6.
• Encoder freeze schedule: encoder frozen for the first 2 epochs (train the CTC head only), then unfrozen for full end-to-end adaptation — the main anti-forgetting lever.
• Batching: batch 8 × grad-accum 8 (effective ~64), bf16-mixed, max_duration 16 s.
• Stopping: validate each epoch, early-stop on dev val_wer (patience 8); best at epoch 14, dev WER 13.59 %. ~23 epochs / ~5.1 h on one NVIDIA RTX A4500 (20 GB).

---

Limitations & caveats

• Pseudo-labels. Training/eval transcripts are produced by an alignment pipeline (whisper + parakeet-CTC), not human annotation (train corpus WER ~17.7 % vs the rnnt judge). The model is only as good as those labels.
• Circularity. Because the labels are partly CTC-derived, a CTC fine-tune can look better than it is when scored against them. The rnnt-1.1b judge (20.03 % base) is the independent reference; the fine-tune beating it is the trustworthy signal.
• Domain. Tuned for Indian legal/court English (accents, legal vocabulary). Expect smaller or no gains on general or out-of-domain speech.
• Format. 16 kHz mono; lowercase, unpunctuated output.

License & attribution

• Weights derive from nvidia/parakeet-ctc-0.6b, released under CC-BY-4.0 — this fine-tune is distributed under the same license, with attribution to NVIDIA.
• The training/evaluation data (vanarp/legal2023_38hrs) is restricted court-proceeding audio; using this model or reproducing these numbers is subject to that dataset's terms. Ensure you have the right to use the data accordingly.

Citation

Fine-tuned by Pranav (vanarp). Base model: NVIDIA NeMo Parakeet-CTC-0.6B. Pipeline & method: alignment → speaker-segment pseudo-labels → CTC fine-tune (see the dataset card).