ainotes-whisper-marathi-q5_1

Whisper Marathi (small) — GGML q5_1

A GGML conversion of steja/whisper-small-marathi, quantised to q5_1 for on-device use with whisper.cpp.

190 MB. Runs faster than real time on a 2020 mid-range phone (Snapdragon 720G, 4 threads).

Measured

Scored on 32 FLEURS mr_in clips through whisper.cpp itself, with the same decoding settings the application uses — greedy, no_timestamps.

Scoring romanises both reference and hypothesis to a common script first, so a legitimate spelling variant is not counted as a whole-word error.

Required setting

Set no_timestamps. Indian-language Whisper fine-tunes are trained on plain transcripts with timestamp prediction disabled, leaving their timestamp tokens untrained. whisper.cpp trusts those tokens to decide where a segment ends, so with them enabled it cuts in the wrong place and the decoder stops following the audio — emitting fluent, grammatical, entirely unrelated text.

Measured on Hindi across 64 clips: 47.5% WER with timestamps on, 14.9% with them off. single_segment is not a substitute; it still lets the model emit timestamp tokens.

whisper-cli -m ggml-model.bin -f audio.wav -l mr -nt

Conversion

Hugging Face → GGML float16 via whisper.cpp's convert-h5-to-ggml.py, then quantised to q5_1. The token table is verified byte for byte against ggerganov's published ggml-small; a mismatched vocabulary produces fluent, confident, wrong output that nothing else catches.

q5_1 measured 2.6× smaller, 1.34× faster and no less accurate than float16 (14.7% vs 15.9% WER on Hindi, same clips).

Licence and attribution

Derived from steja/whisper-small-marathi, licensed apache-2.0, and released under the same terms. The underlying Whisper architecture is MIT (OpenAI). Credit for the fine-tuning belongs to the source authors; this repository contributes only the format conversion.