typhoon-asr-realtime-nemo-ctc

typhoon-asr-realtime-nemo-ctc

A Thai CTC model built entirely from standard NeMo classes built on typhoon-ai/typhoon-asr-realtime: EncDecCTCModelBPE whose ConformerEncoder carries the frozen 17 RT layers plus 4 newly trained layers (26M trainable) and a ConvASRDecoder over the RT BPE-2048 tokenizer. Loads with restore_from anywhere NeMo is installed; using only standard NeMo modules keeps the usual NeMo/Riva/TensorRT export paths.

Results

Plain transcription (CER%, same protocol: gigaspeech2 bench in-domain / TVSpeech out-of-domain 30-s talk-show):

† dominated by 3 repetition-loop utterances; excluding them: 8.39.

Read this model's CER in role: it trades a few CER points against the larger autoregressive decoders for frame-level CTC posteriors — the thing that enables keyword spotting, word boosting, and timestamps below — while adding only 26M parameters on top of the typhoon-asr-realtime encoder.

Trained on ~10,800 hours of Thai speech (GigaSpeech2-th); see the Typhoon ASR Real-time technical report.

Contextual biasing — vs every route we measured

Same benchmark (wayu-ai/thai-contextasr-bench), same frozen scorer. Cells = CER% ↓ / entity recall ↑. Whisper models take the list as a decoder prompt (zero-shot); typhoon-asr-qwen-1.7b-ctx is a LoRA fine-tuned for biasing; this model spots and splices at decode time (word_boost.py, τ=−2).

N/A: typhoon-whisper's prompt pathway collapses on any list (echo loops, CER 200+ — erased by its prompt-free fine-tune); the fine-tuned medium and Pathumma/Thonburian cannot take lists beyond ~10–15 entries because Whisper's 448-token decoder window is shared between the prompt and the transcript. This model and typhoon-asr-qwen-1.7b-ctx have no list-size limit; only they remain standing past bias@10 — and this model's recall stays flat (.85 → .76 from 0 to 500 distractors) where every other route either dies or decays.

Rule of thumb: pick this model when entity recall, list-size robustness, compute, or streaming matter — it finds more of your keywords at ~1/15 the size with zero biasing training. Pick typhoon-asr-qwen-1.7b-ctx when absolute CER and tolerance to huge dirty lists matter most. They also compose: typhoon-qwen writes the transcript, this model supplies keyword hits and timestamps from one cheap encoder pass.

Usage

from nemo.collections.asr.models import EncDecCTCModelBPE
m = EncDecCTCModelBPE.restore_from("typhoon-asr-realtime-nemo-ctc.nemo")
print(m.transcribe(["clip.wav"])[0].text)

Keyword spotting on the CTC posteriors (bundled kws_spot.py):

python kws_spot.py --model typhoon-asr-realtime-nemo-ctc.nemo \
    --audio clip.wav --keywords "วโรรส,เฟซบุ๊ก"

Word boosting (bundled word_boost.py — spot then splice into the transcript, threshold --tau):

python word_boost.py --model typhoon-asr-realtime-nemo-ctc.nemo \
    --audio clip.wav --keywords "วโรรส,เฟซบุ๊ก" --tau -1.5

On thai-contextasr-bench (gold list): CER 9.56 → 5.97 with entity recall .158 → .852 at tau −2 (FA .003); tau −1.5 balanced (recall .758, FA .0014). Keep lists curated (≤50 entries) at aggressive tau.

Reference

Typhoon ASR: see the Typhoon ASR Real-time technical report.

License & attribution

CC-BY-4.0 — this checkpoint embeds encoder weights of typhoon-ai/typhoon-asr-realtime (CC-BY-4.0) verbatim; attribution to the Typhoon team is required. Newly trained layers and bundled scripts follow the same license.

Acknowledgements

We thank Kunat Pipatanakul and Potsawee Manakul (wayu-ai) for their valuable feedback.