typhoon-whisper-large-v3-ctc

typhoon-whisper-large-v3-ctc

Word- and character-level timestamps for Thai: a 29M character-CTC head on the frozen typhoon-ai/typhoon-whisper-large-v3 encoder (pinned revision 748e8a4, recorded in config.json). One encoder pass gives transcription and alignment.

It is both an aligner and a compact ASR model: it force-aligns given text (a transcript or ASR hypothesis) to word/character timestamps, and its greedy CTC decode transcribes directly (--transcribe, no text needed).

Results

Trained on 1,000 h of GigaSpeech2-th.

Usage

pip install torch torchaudio transformers safetensors huggingface_hub soundfile pythainlp
python standalone_align.py --head typhoon-ai/typhoon-whisper-large-v3-ctc \
    --audio clip.wav --text "ข้อความ" --words

# ASR mode (greedy CTC decode)
python standalone_align.py --head typhoon-ai/typhoon-whisper-large-v3-ctc --audio clip.wav --transcribe

standalone_align.py (bundled) has no other dependencies; the frozen encoder downloads automatically at the pinned revision. Files: head.safetensors, config.json (architecture + encoder id/revision), ctc_vocab.json (80-symbol Thai char vocab v1.0.0).

How it works

Frozen Whisper encoder (50 Hz) → 29M Transformer head → per-frame character posteriors → CTC forced alignment against the given text → character spans on a 20 ms grid; words grouped with pythainlp newmm or caller-supplied segmentation.

Reference

Typhoon ASR: see the Typhoon ASR Real-time technical report.

Acknowledgements

We thank Kunat Pipatanakul and Potsawee Manakul (wayu-ai) for their valuable feedback.