Downloads · 30 days
12
9% of all-time downloads
kiviki/whisper-turbo-romani-lora
whisper-turbo-romani-lora is a automatic speech recognition model from kiviki. Use it when you need speech turned into text. It is set up for peft. The card lists the license as mit.
Status: experimental Whisper adapter baseline. This adapter is kept for reproducibility and comparison. In the current frozen local evaluation, zero-shot facebook/mms-1b-all with rmc-scriptlatin performs better on bot…
Downloads · 30 days
12
9% of all-time downloads
All-time downloads
131
Public
Repo size
1.3 MB
Likes
0
Public
Click a slice to open those files.
.json3.9 MB · 75%
From the Hugging Face model README
Status: experimental Whisper adapter baseline. This adapter is kept for reproducibility and comparison. In the current frozen local evaluation, zero-shot
facebook/mms-1b-allwithrmc-script_latinperforms better on both evaluated splits, so this repository should not be read as the strongest current Romani ASR result.
This repository contains a PEFT LoRA adapter for
openai/whisper-large-v3-turbo, fine-tuned for Latin-script Romani ASR.
This is not a standalone Whisper checkpoint. Load the base model from Hugging Face, then load this adapter and the tokenizer/processor files from this repository.
This package is a slim adapter export. It omits redundant full base
embedding/projection matrices and keeps the LoRA weights plus the trained
<|romani|> token deltas. Before loading the adapter, initialize the custom
<|romani|> token from Whisper's Slovak token (<|sk|>), then force the
Romani decoder prompt.
The training/evaluation harness that produced this adapter contains the required loader path:
python scripts/evaluate_hf_asr.py \
--model-id openai/whisper-large-v3-turbo \
--adapter-dir kiviki/whisper-turbo-romani-lora \
--processor-dir kiviki/whisper-turbo-romani-lora \
--language slovak \
--task transcribe \
--new-language-token romani \
--max-length 72 \
--no-repeat-ngram-size 3 \
--repetition-penalty 1.15
openai/whisper-large-v3-turboq_proj, v_proj<|romani|><|romani|><|transcribe|><|notimestamps|>All results below were rerun on 2026-08-10 with fixed manifests and guarded decoding. Lower WER/CER is better.
Naming used below:
test.csv, the original single-source prompt set
(R_prompts).validation_clean.csv, the cleaned multi-source
validation split.facebook/mms-1b-all with
target_lang=rmc-script_latin, with no local MMS training.| Split | System | Role | WER | CER | ASCII WER | ASCII CER | Non-Latin |
|---|---|---|---|---|---|---|---|
| Prompt Test | Whisper Turbo Baseline | unadapted baseline | 0.796 | 0.184 | 0.719 | 0.138 | 0 |
| Prompt Test | Whisper Turbo Romani LoRA | trained Whisper adapter | 0.653 | 0.122 | 0.606 | 0.101 | 0 |
| Prompt Test | MMS-1B-All Romani/Latin Zero-Shot | external zero-shot comparison | 0.300 | 0.047 | 0.293 | 0.046 | 0 |
| Multi-Recording Dev | Whisper Turbo Baseline | unadapted baseline | 0.921 | 0.496 | 0.882 | 0.459 | 0 |
| Multi-Recording Dev | Whisper Turbo Romani LoRA | trained Whisper adapter | 0.769 | 0.405 | 0.753 | 0.393 | 0 |
| Multi-Recording Dev | MMS-1B-All Romani/Latin Zero-Shot | external zero-shot comparison | 0.756 | 0.383 | 0.746 | 0.376 | 0 |
Prompt Test contains one source group only, so it is useful for continuity with the original model card but too easy to be the only headline result. Multi-Recording Dev spans five recording groups and is a better stress test for iteration.
The detailed frozen report is included in this repository at
eval/frozen-asr-evaluation-2026-08-10.md.
This is an experimental low-resource ASR adapter. It still makes frequent word-boundary, spelling, and diacritic mistakes. It is kept as a reproducible Whisper adapter baseline while MMS adapter training is investigated separately.
The MMS rows above are comparison results only. This repository does not contain an MMS adapter.
Use this adapter for experimentation with Latin-script Romani transcription. Review transcripts before using them in user-facing or high-stakes settings.
This repository does not include the base Whisper model, raw audio data, training optimizer state, or training/evaluation logs.