whisper-large-v3-turbo-arabic-ft-lora

Whisper-large-v3-turbo Arabic FT — LoRA adapter

LoRA adapter trained on dialect-balanced Arabic (4 dialects: MSA + Egyptian + Levantine + Gulf, 17.2k rows). Lightweight (~111 MB). Use this if you want to continue fine-tuning on top of the same base, or merge it for inference.

For ready-to-use deployment artifacts (no PEFT dependency required):

• merged HuggingFace model: dev-ahmedhany/whisper-large-v3-turbo-arabic-ft
• production CTranslate2 / faster-whisper: dev-ahmedhany/whisper-large-v3-turbo-arabic-ft-ct2-int8

Performance: zero-shot baseline vs. fine-tuned

Held-out test sets, n=100 per dialect, beam=1, deterministic Arabic normalizer (src/normalization.py). Zero-shot baseline is the unmodified openai/whisper-large-v3-turbo (CT2 int8 / CPU). Fine-tuned numbers are this model evaluated as PEFT bf16 on a single L4 GPU (apples-to-apples with training precision).

The val WER during training reaches 33.10% on a held-out slice of the training-source distribution.

Dialects scoped out of v1

• Maghrebi (Moroccan/Algerian) — excluded from training and reporting. Whisper has insufficient Moroccan Arabic in pretraining (84.7% zero-shot WER at large-v3 int8); QLoRA cannot recover within this budget.
• Levantine — included in training (MASC, ~4 h broadcast TV) but excluded from the headline. The held-out Casablanca Levantine test set has very different acoustic characteristics from MASC (mixed-genre / phone-quality vs broadcast studio); the v1 model overfit MASC's narrow distribution. Reported FT WER 51.9% vs zero-shot 40.3% — that gap is a train-test domain mismatch, not a Levantine modeling failure. The v2 retrain (r=8, α=16, + Casablanca train splits) is designed to fix this.

Use cases for v1

• ✅ Egyptian and Gulf Arabic, mixed-genre / phone-quality audio — clean 2–3 pp lift over zero-shot
• ⚠️ MSA only — use unmodified openai/whisper-large-v3-turbo (this model regresses ~1 pp on FLEURS MSA)
• ⚠️ Levantine — wait for v2 (in progress); v1 overfits MASC broadcast and underperforms on out-of-domain Levantine

Training recipe

• Base: openai/whisper-large-v3-turbo
• Method: QLoRA (NF4 4-bit base, double-quant, bf16 compute)
• LoRA: r=32, alpha=64, target = q/k/v/out_proj + fc1/fc2, dropout=0.05
• Optimizer: paged_adamw_8bit, lr=1e-4, warmup ratio 0.1
• Schedule: 3 epochs, effective batch 16, eval/save every 500 steps
• Hardware: single L4 24GB GPU, ~7h training time

Validation WER (during training)

n=907 (dialect-balanced val mix, no Maghrebi)

Data sources (no Maghrebi)

Maghrebi excluded because Whisper's pretraining has insufficient Moroccan/Algerian Arabic (84.7% zero-shot WER at large-v3 int8); QLoRA cannot bring it within range of other dialects in this pipeline.

Usage

from peft import PeftModel
from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch

base = WhisperForConditionalGeneration.from_pretrained(
    "openai/whisper-large-v3-turbo", torch_dtype=torch.bfloat16
)
processor = WhisperProcessor.from_pretrained(
    "openai/whisper-large-v3-turbo", language="arabic", task="transcribe"
)
model = PeftModel.from_pretrained(base, "dev-ahmedhany/whisper-large-v3-turbo-arabic-ft-lora")

# inference (or continue training first)
# model = model.merge_and_unload()  # for inference; not required for further FT

Citation

@misc{hany2026whisper-arabic-dialects,
  author       = {Hany, Ahmed},
  title        = {Production-Grade Arabic Multi-Dialect ASR via QLoRA Fine-Tuning of Whisper},
  year         = {2026},
  url          = {https://github.com/dev-ahmedhany/whisper-arabic-dialects},
  orcid        = {0009-0000-8756-9520}
}

• Repo: https://github.com/dev-ahmedhany/whisper-arabic-dialects
• Author: Ahmed Hany (ORCID 0009-0000-8756-9520)