whisper-large-v3-turbo-arabic-ft

Whisper-large-v3-turbo Arabic FT — merged model

Standalone HuggingFace model (no PEFT dependency required). Adapter has been merged into base weights — load directly with transformers. For further fine-tuning prefer the LoRA adapter.

Performance: zero-shot baseline vs. fine-tuned

Held-out test sets, n=100 per dialect, beam=1, deterministic Arabic normalizer (src/normalization.py). Zero-shot baseline is the unmodified openai/whisper-large-v3-turbo (CT2 int8 / CPU). Fine-tuned numbers are this model evaluated as PEFT bf16 on a single L4 GPU (apples-to-apples with training precision).

The val WER during training reaches 33.10% on a held-out slice of the training-source distribution.

Dialects scoped out of v1

• Maghrebi (Moroccan/Algerian) — excluded from training and reporting. Whisper has insufficient Moroccan Arabic in pretraining (84.7% zero-shot WER at large-v3 int8); QLoRA cannot recover within this budget.
• Levantine — included in training (MASC, ~4 h broadcast TV) but excluded from the headline. The held-out Casablanca Levantine test set has very different acoustic characteristics from MASC (mixed-genre / phone-quality vs broadcast studio); the v1 model overfit MASC's narrow distribution. Reported FT WER 51.9% vs zero-shot 40.3% — that gap is a train-test domain mismatch, not a Levantine modeling failure. The v2 retrain (r=8, α=16, + Casablanca train splits) is designed to fix this.

Use cases for v1

• ✅ Egyptian and Gulf Arabic, mixed-genre / phone-quality audio — clean 2–3 pp lift over zero-shot
• ⚠️ MSA only — use unmodified openai/whisper-large-v3-turbo (this model regresses ~1 pp on FLEURS MSA)
• ⚠️ Levantine — wait for v2 (in progress); v1 overfits MASC broadcast and underperforms on out-of-domain Levantine

Training recipe

• Base: openai/whisper-large-v3-turbo
• Method: QLoRA (NF4 4-bit base, double-quant, bf16 compute), then merged → fp32
• LoRA: r=32, alpha=64, target = q/k/v/out_proj + fc1/fc2, dropout=0.05
• Optimizer: paged_adamw_8bit, lr=1e-4, warmup ratio 0.1
• Schedule: 3 epochs, effective batch 16, eval/save every 500 steps
• Hardware: single L4 24GB GPU, ~7h training time

Validation WER (during training)

n=907 (dialect-balanced val mix, no Maghrebi)

Data sources (no Maghrebi)

Maghrebi excluded — Whisper has insufficient Moroccan/Algerian Arabic in pretraining (84.7% zero-shot WER at large-v3 int8) for QLoRA to bring it within range of other dialects in this pipeline.

Usage

from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch

model = WhisperForConditionalGeneration.from_pretrained(
    "dev-ahmedhany/whisper-large-v3-turbo-arabic-ft", torch_dtype=torch.bfloat16
)
processor = WhisperProcessor.from_pretrained("dev-ahmedhany/whisper-large-v3-turbo-arabic-ft", language="arabic", task="transcribe")

# inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
# ids = model.generate(input_features=inputs.input_features, language="arabic", task="transcribe")
# text = processor.batch_decode(ids, skip_special_tokens=True)

Citation

@misc{hany2026whisper-arabic-dialects,
  author       = {Hany, Ahmed},
  title        = {Production-Grade Arabic Multi-Dialect ASR via QLoRA Fine-Tuning of Whisper},
  year         = {2026},
  url          = {https://github.com/dev-ahmedhany/whisper-arabic-dialects},
  orcid        = {0009-0000-8756-9520}
}

• Repo: https://github.com/dev-ahmedhany/whisper-arabic-dialects
• Author: Ahmed Hany (ORCID 0009-0000-8756-9520)