whisper-physio-lora

Whisper Small + LoRA on German Physiotherapy Dictations

Fine-tuned LoRA adapter for openai/whisper-small on German physiotherapy treatment dictations.

Results

Evaluated on 10 held-out German physio dictations:

Relative WER improvement over baseline Small: 19% — approaching the quality of Whisper Medium at 1/3 the model size.

Training

• Base model: openai/whisper-small (244M parameters)
• Method: LoRA (r=8, alpha=32) on q_proj and v_proj
• Trainable parameters: 884,736 (0.36% of total)
• Dataset: 40 self-recorded German physiotherapy dictations
• Val set: 10 additional dictations
• Epochs: 10
• Learning rate: 1e-4 with linear warmup (20 steps) and decay
• Batch size: 4
• Precision: fp16
• Hardware: NVIDIA GTX 1650 Ti (4GB VRAM)

Usage

from transformers import WhisperProcessor, WhisperForConditionalGeneration
from peft import PeftModel
import librosa

processor = WhisperProcessor.from_pretrained("toniju98/whisper-physio-lora")
base_model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small")
model = PeftModel.from_pretrained(base_model, "toniju98/whisper-physio-lora")

audio, _ = librosa.load("dictation.wav", sr=16000)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
predicted_ids = model.generate(inputs.input_features, language="de", task="transcribe")
transcript = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcript)

Limitations

• Very small validation set (10 samples with 20 domain terms) — results should be considered indicative, not statistically significant
• Single speaker — trained on one voice, may generalize poorly to other speakers
• Specific to physiotherapy vocabulary — not a general-purpose German ASR improvement
• LoRA overhead — adds ~2ms per token during inference vs. baseline

Author

Portfolio project by Antonio Juric (GitHub). Part of the PhysioDoc pipeline: audio → transcript → structured treatment documentation.