whisper-ja-760M

Whisper finetune for Japanese focused on general/anime domains. For usage instructions follow openai/whisper-large-v3-turbo. Due to vocab changes ctranslate2>=4.7.1 required for faster-whisper. For inference engines with hardcoded vocab, the token embedding can be padded.

Finetuned from turbo with pruned vocab, indices can be found in mapping.txt. Trained decoder only for 2^20 steps, batch size 64. Using a 45000 hour corpus (largest source is 17000 of filtered reazonspeech-all) with custom mixing ratio and augmentation to maintain long form performance and timestamps.

Benchmarks. Competitive/SOTA on test sets, slightly better than 1.5B on short form, worse on long form. Also trained for lyrics but untested.

Currently training 760M pruned to 488M with LiteASR. Likely to be final model due to diminishing improvements I can make and compute limitations.

Acknowledgements

• Train sets: OOPPEENN, Reazon, 小虫哥_, Common Voice 20, deepghs
• Test sets: KitsuneX07, TEDxJP, kotoba-tech, Saruwatari-lab, grider-withourai