Downloads · 30 days
279
56% of all-time downloads
gcoli/whisper-large-v3-turbo-swiss-german-mit
whisper-large-v3-turbo-swiss-german-mit is a automatic speech recognition model from gcoli. Use it when you need speech turned into text. It is set up for transformers. The card lists the license as mit.
Swiss German speech to Standard German text. LoRA fine-tune of openai/whisper-large-v3-turbo, merged into the weights, with a deliberately clean licence chain: MIT base model, MIT training data, MIT result.
Downloads · 30 days
279
56% of all-time downloads
All-time downloads
498
Public
Parameters
809M
3.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.bin1.6 GB · 50%
From the Hugging Face model README
Swiss German speech to Standard German text. LoRA fine-tune of
openai/whisper-large-v3-turbo, merged into the weights, with a deliberately clean licence
chain: MIT base model, MIT training data, MIT result.
Most public Swiss German models inherit CC BY-NC from SwissDial and cannot be used commercially. This checkpoint avoids that: it is trained only on MIT-licensed FHNW/i4ds material.
Measured on the FHNW/i4ds All Swiss German Dialects Test Set (MIT, 5750 clips, 12.7 h, 17 dialects), decoded with faster-whisper (CTranslate2, float16, beam 5) on an RTX 3080 Ti.
| Model | WER dev | WER final | CER | RTF |
|---|---|---|---|---|
| this checkpoint | 23.7 % | 25.9 % | 13.4 % | 0.033 |
openai/whisper-large-v3-turbo (no fine-tune) | 26.5 % | 28.5 % | 14.6 % | 0.036 |
openai/whisper-large-v3 (no fine-tune) | 24.0 % | 26.3 % | 13.7 % | 0.080 |
2.8 WER points better than the untouched base model at identical speed.
The test set is split speaker-disjointly into two halves: dev (2930 clips) was used for model selection, final (2820 clips) was held out. The final half is harder for every model measured, including untrained baselines — compare within a column, not across. The test set was never trained on.
WER/CER are per-clip means after normalisation (lowercase, punctuation removed, ß → ss).
faster-whisper / CTranslate2 — weights are in the ct2/ subfolder, so fetch that folder
first (the repository root holds the transformers format, whose config.json differs):
from huggingface_hub import snapshot_download
from faster_whisper import WhisperModel
path = snapshot_download("gcoli/whisper-large-v3-turbo-swiss-german-mit",
allow_patterns="ct2/*")
model = WhisperModel(f"{path}/ct2", device="cuda", compute_type="float16")
segments, info = model.transcribe("input.wav", language="de", beam_size=5)
print(" ".join(s.text.strip() for s in segments))
transformers — the repository root:
from transformers import pipeline
asr = pipeline("automatic-speech-recognition",
model="gcoli/whisper-large-v3-turbo-swiss-german-mit", device=0)
print(asr("input.wav", generate_kwargs={"language": "german", "task": "transcribe"})["text"])
Use language token de and task transcribe: the model outputs Standard German, not dialect
spelling. Audio should be 16 kHz mono.
On recordings of several minutes, turn context carry-over off. faster-whisper leaves
condition_on_previous_text on by default, and on 15-minute files it cost this model family
2.5 (turbo) to 11.7 (large) WER points in our measurements, with the output growing ~14 %
longer than the reference — the model starts inventing text. VAD filtering did not help either.
segments, info = model.transcribe("meeting.wav", language="de", beam_size=5,
condition_on_previous_text=False, vad_filter=False)
Measured on 26 files of ~15 min each, built by concatenating the held-out half of the test set, so the audio content is identical to the short-form numbers above. Expect roughly 1 to 3 WER points worse on long recordings than on single sentences; that gap differs per model and does not follow the short-form ranking.
ct2/int8_float16 quantisation costs about 0.3 WER points on this checkpoint's base family and
is a reasonable trade for VRAM-constrained setupsmlx_whisper| Base | openai/whisper-large-v3-turbo (MIT) |
| Method | LoRA r=32, α=64, target q_proj/v_proj, dropout 0.05, merged after training |
| Optimiser | lr 3e-4, linear decay, 500 warmup steps, 1 epoch, batch 16, bf16 + gradient checkpointing |
| Data | 40 h FHNW/i4ds Swiss Parliaments Corpus v2 (Bern) + 40 h Gemeinderat Zürich |
| Hardware | single RTX 3080 Ti (12 GB), ~1.6 h training |
The Zürich corpus is audio only. Transcripts were generated with openai/whisper-large-v3
(Apache-2.0) and filtered by confidence (avg_logprob > -0.45, no_speech_prob < 0.3,
compression ratio < 2.2, repetition-loop rejection).
Two findings from the run series this checkpoint comes from:
Fine-tuning on one language variety can damage the others. Measured on google/fleurs
(250 validation clips per language, no prompt, beam 5):
| German | English | French | Italian | Spanish | |
|---|---|---|---|---|---|
| this checkpoint | 5.2 % | 5.7 % | 5.5 % | 3.4 % | 3.9 % |
openai/whisper-large-v3-turbo | 4.9 % | 4.9 % | 5.2 % | 2.4 % | 3.2 % |
The fine-tune costs 0.3 to 1.0 WER points in the other languages — a real but mild regression, not catastrophic forgetting. If you need a multilingual system, keep the base model for everything except Swiss German.
openai/whisper-large-v3-turboopenai/whisper-large-v3 (Apache-2.0)training_provenance.jsonMIT. The base model (openai/whisper-large-v3-turbo) is MIT, the training data (FHNW/i4ds
Swiss Parliaments, Gemeinderat Zürich) is MIT, and the pseudo-label teacher
(openai/whisper-large-v3) is Apache-2.0. Commercial use is permitted.
Please credit FHNW/i4ds for the underlying corpora. This repository grants no additional rights to upstream models, training data or software, and implies no endorsement by FHNW, i4ds or OpenAI.