Downloads · 30 days
354
9% of all-time downloads
whisper-large-v3-turbo-sq-v2
Flutra/whisper-large-v3-turbo-sq-v2
whisper-large-v3-turbo-sq-v2 is a machine learning model from Flutra. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a fine-tuned version of the Whisper Large v3 Turbo model, optimized for Albanian speech-to-text transcription. It achieves a Word Error Rate (WER) of 6.98% on a held-out evaluation set.
All-time downloads
3.9K
Public
Parameters
809M
22.6 GB on disk
Likes
8
Public
Repo makeup
Click a slice to open those files.
.safetensors3.2 GB · 100%
At a glance
- Model type
- whisper
- Access
- Public
- Created
- Mar 18, 2025
- Updated
- Mar 20, 2025
- SHA
- 2e7b490e
Model card
From the Hugging Face model README
Whisper Large v3 Turbo (Albanian Fine-Tuned) - v2
This is a fine-tuned version of the Whisper Large v3 Turbo model, optimized for Albanian speech-to-text transcription. It achieves a Word Error Rate (WER) of 6.98% on a held-out evaluation set.
Model Details
- Base Model:
openai/whisper-large-v3-turbo - Language: Albanian (
sq)
Training Dataset
- Source: Mozilla Common Voice version 19 (available in HF as Kushtrim/common_voice_19_sq)
- Description: Audio clips ranging from 5-30 seconds, in spoken Albanian.
Training Details
The model was fine-tuned on an NVIDIA A100 GPU (40GB) using the transformers library. Below are the key training arguments:
| Argument | Value | Description |
|---|---|---|
per_device_train_batch_size | 8 | Training batch size per GPU |
per_device_eval_batch_size | 2 | Evaluation batch size per GPU |
gradient_accumulation_steps | 1 | Steps to accumulate gradients (effective batch size = 8) |
num_train_epochs | 3 | Number of training epochs |
learning_rate | 1e-5 | Initial learning rate |
warmup_steps | 300 | Number of warmup steps for learning rate |
evaluation_strategy | "steps" | Evaluate every eval_steps during training |
eval_steps | 250 | Frequency of evaluation (every 250 steps) |
fp16 | True | Use mixed precision training (16-bit floats) |
- Total Steps: ~3,540 (completed 3,500)
- Hardware: NVIDIA A100 (40GB)
- Libraries:
- transformers==4.38.2
- torch==2.2.1
Performance
| Step | Training Loss | Validation Loss | WER |
|---|---|---|---|
| 250 | 0.4744 | 0.3991 | 34.03% |
| 500 | 0.3421 | 0.3426 | 30.42% |
| 750 | 0.2871 | 0.2808 | 26.09% |
| 1000 | 0.2401 | 0.2258 | 21.31% |
| 1250 | 0.1809 | 0.1998 | 19.15% |
| 1500 | 0.1142 | 0.1827 | 17.33% |
| 1750 | 0.1051 | 0.1611 | 15.19% |
| 2000 | 0.0930 | 0.1464 | 13.82% |
| 2250 | 0.0827 | 0.1313 | 11.79% |
| 2500 | 0.0420 | 0.1139 | 10.50% |
| 2750 | 0.0330 | 0.1124 | 9.58% |
| 3000 | 0.0255 | 0.1006 | 8.38% |
| 3250 | 0.0256 | 0.0905 | 7.48% |
| 3500 | 0.0204 | 0.0889 | 6.98% |
- Final WER: 6.98% (at step 3500)
- Type
- whisper
- Created
- Mar 18, 2025
- Updated
- Mar 20, 2025