flix-swissgerman-lora

flix-swissgerman-lora

A LoRA adapter for openai/whisper-large-v3 fine-tuned for Swiss German (Schweizerdeutsch) automatic speech recognition. The adapter transcribes Swiss German dialect speech into grammatically correct Standard German text.

This is among the first publicly available, honestly evaluated LoRA adapters for Swiss German ASR.

📄 Paper: https://arxiv.org/abs/2606.07608

Model Description

• Base model: openai/whisper-large-v3 (1.55B parameters)
• Fine-tuning: LoRA (r=160, α=32, dropout=0.05)
• Trainable parameters: ~1.1B (LoRA weights across q, k, v, out, fc1, fc2)
• Training data: 1,092 hours of Swiss German speech from broadcast subtitles, parliamentary proceedings, and YouTube
• Task: Swiss German speech → Standard German text (dialect-to-standard translation + transcription)
• Hardware: NVIDIA DGX Spark GB10 (128 GB unified memory), single desktop workstation

Performance

Important Context on WER

Our WER of 25.32% should be interpreted carefully:

• ~64% of evaluation samples are semantically correct (KORREKT + STIL categories) but penalized by WER due to transcription convention differences (tense, reformulation style)
• The genuine content error rate is 13.9% cWER; bias-corrected estimation yields 8.5% bWER
• Published lower WER scores (Michaud 17.5%, ZHAW 17.1%) are inflated by benchmark contamination — see our paper for details

Usage

from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel
import torch

base_model_id = "openai/whisper-large-v3"
adapter_id = "Flix-AI/flix-swissgerman-lora"

processor = WhisperProcessor.from_pretrained(base_model_id)
model = WhisperForConditionalGeneration.from_pretrained(
    base_model_id, torch_dtype=torch.float32, device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_id)

# Transcribe Swiss German audio
audio_array = ...  # numpy array, 16kHz mono
input_features = processor(
    audio_array, sampling_rate=16000, return_tensors="pt"
).input_features.to(model.device)

predicted_ids = model.generate(input_features, language="de", task="transcribe")
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

LoRA Configuration

Training Details

Data Sources

No training data is redistributed with this model. The model was trained under the Swiss text and data mining research exception (Art. 24d URG).

Training Configuration

Dialect Coverage

The training data covers all major Swiss German dialect regions:

Limitations

1. Proper nouns: The model may misspell names and places it hasn't encountered during training
2. Word order: Swiss German sentence structure sometimes differs from Standard German; the model may produce valid but differently ordered translations
3. Convention mismatch: Trained on broadcast subtitles (editorial style), which may differ from verbatim transcription expectations
4. No context: The model processes segments independently; it cannot use broader conversation context for disambiguation

Citation

@article{akeret2026whisper-swiss-german,
  title={Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6\% WER (13.8\% cWER)},
  author={Akeret, Felix},
  year={2026},
  url={https://arxiv.org/abs/2606.07608},
  eprint={2606.07608},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

Acknowledgments

• OpenAI for the Whisper model
• FHNW/i4ds for the Swiss Parliament Corpus (SPC v2) and ASGDTS benchmark
• SRF for publicly accessible broadcast content
• PlaySuisse for Swiss film and series content