Downloads · 30 days
15
20% of all-time downloads
whisper-small-en-finetuned
ronxldwilson/whisper-small-en-finetuned
whisper-small-en-finetuned is a automatic speech recognition model from ronxldwilson. Use it when you need speech turned into text. The card lists the license as apache-2.0.
Fine-tuned openai/whisper-small.en using LoRA (PEFT) on English speech data from Google FLEURS (enus).
All-time downloads
76
Public
Parameters
242M
2.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors967 MB · 99%
At a glance
- Task
- Automatic Speech Recognition
- License
- apache-2.0
- Model type
- whisper
- Access
- Public
- Created
- Jun 1, 2026
- Updated
- Jun 1, 2026
- SHA
- 93f8b7da
Base models
Datasets
Model card
From the Hugging Face model README
Whisper Small.en — Fine-tuned
Fine-tuned openai/whisper-small.en using LoRA (PEFT) on English speech data from Google FLEURS (en_us).
Built for the s2st-en-hi real-time speech-to-speech translation pipeline (English ↔ Indic languages).
Training details
| Base model | openai/whisper-small.en (244M params) |
| Method | LoRA (r=16, alpha=32) — 5.6M trainable / 247M total (2.3%) |
| Data | 1,800 train / 200 val clips from FLEURS en_us |
| Epochs | 3 |
| Batch size | 2 x 8 gradient accumulation = 16 effective |
| Learning rate | 1e-4 with warmup (50 steps) |
| Device | Apple M4 (MPS), 29 min total |
| Peak RAM | ~5.2 GB |
Results
| Epoch | WER | Eval Loss |
|---|---|---|
| 1 | 7.2% | 0.214 |
| 2 | 5.9% | 0.192 |
| 3 | 5.76% | 0.192 |
A/B test on 15 TTS-synthesized English phrases:
- Baseline
small.en: 7/15 (47%) exact match - Fine-tuned: 12/15 (80%) exact match
- Latency: identical (~1.4s/utterance on CPU int8)
Usage
from transformers import WhisperForConditionalGeneration, WhisperProcessor
model = WhisperForConditionalGeneration.from_pretrained("ronxldwilson/whisper-small-en-finetuned")
processor = WhisperProcessor.from_pretrained("ronxldwilson/whisper-small-en-finetuned")
For the CTranslate2 int8 version (production, faster-whisper): see ronxldwilson/whisper-small-en-finetuned-ct2-int8
- Task
- Automatic Speech Recognition
- Type
- whisper
- License
- apache-2.0
- Languages
- en
- Created
- Jun 1, 2026
- Updated
- Jun 1, 2026