whisper-large-v3-turbo-fa

Whisper Large V3 Turbo - Persian Fine-Tuned

This repository contains a Persian (Farsi) fine-tuned version of OpenAI Whisper Large V3 Turbo for automatic speech recognition (ASR).

Model Information

• Model: sadrasa97/whisper-large-v3-turbo-fa
• Base Model: openai/whisper-large-v3-turbo
• Task: Automatic Speech Recognition
• Language: Persian / Farsi
• Language Code: fa
• Architecture: Whisper
• Framework: Hugging Face Transformers

Installation

pip install -U torch transformers accelerate librosa soundfile

## Quick Start

The recommended way to use the model is with the Hugging Face `pipeline` API.

```python
import torch

from transformers import (
    AutoModelForSpeechSeq2Seq,
    AutoProcessor,
    pipeline,
)

model_id = "sadrasa97/whisper-large-v3-turbo-fa"

device = "cuda:0" if torch.cuda.is_available() else "cpu"

torch_dtype = (
    torch.float16
    if torch.cuda.is_available()
    else torch.float32
)

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=torch_dtype,
    low_cpu_mem_usage=True,
)

model.to(device)

processor = AutoProcessor.from_pretrained(
    model_id
)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch_dtype,
    device=0 if torch.cuda.is_available() else -1,
)

result = pipe(
    "audio.wav",
    generate_kwargs={
        "language": "fa",
        "task": "transcribe",
    },
)

print(result["text"])

GPU Inference

For NVIDIA CUDA GPUs, FP16 inference can be used to reduce memory consumption and improve inference performance.

import torch

from transformers import (
    AutoModelForSpeechSeq2Seq,
    AutoProcessor,
    pipeline,
)

model_id = "sadrasa97/whisper-large-v3-turbo-fa"

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True,
)

model.to("cuda")

processor = AutoProcessor.from_pretrained(
    model_id
)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch.float16,
    device=0,
)

result = pipe(
    "audio.wav",
    generate_kwargs={
        "language": "fa",
        "task": "transcribe",
    },
)

print(result["text"])

CPU Inference

The model can also be used on CPU.

import torch

from transformers import (
    AutoModelForSpeechSeq2Seq,
    AutoProcessor,
    pipeline,
)

model_id = "sadrasa97/whisper-large-v3-turbo-fa"

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=torch.float32,
    low_cpu_mem_usage=True,
)

processor = AutoProcessor.from_pretrained(
    model_id
)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch.float32,
    device=-1,
)

result = pipe(
    "audio.wav",
    generate_kwargs={
        "language": "fa",
        "task": "transcribe",
    },
)

print(result["text"])

Direct Model Usage

For applications requiring lower-level control over preprocessing and generation, the model can be loaded directly.

import torch
import librosa

from transformers import (
    AutoModelForSpeechSeq2Seq,
    AutoProcessor,
)

model_id = "sadrasa97/whisper-large-v3-turbo-fa"

device = "cuda" if torch.cuda.is_available() else "cpu"

torch_dtype = (
    torch.float16
    if device == "cuda"
    else torch.float32
)

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=torch_dtype,
    low_cpu_mem_usage=True,
)

model.to(device)
model.eval()

processor = AutoProcessor.from_pretrained(
    model_id
)

audio, sample_rate = librosa.load(
    "audio.wav",
    sr=16000,
)

inputs = processor(
    audio,
    sampling_rate=sample_rate,
    return_tensors="pt",
)

input_features = inputs.input_features.to(device)

with torch.inference_mode():

    predicted_ids = model.generate(
        input_features,
        language="fa",
        task="transcribe",
    )

transcription = processor.batch_decode(
    predicted_ids,
    skip_special_tokens=True,
)

print(transcription[0])

Persian Transcription

For Persian speech recognition, explicitly specify the language as Persian:

generate_kwargs = {
    "language": "fa",
    "task": "transcribe",
}

The task should be set to transcribe when the goal is to produce text in the original spoken language.

For example:

result = pipe(
    "persian_audio.wav",
    generate_kwargs={
        "language": "fa",
        "task": "transcribe",
    },
)

print(result["text"])

Audio Input

Whisper operates on audio sampled at 16 kHz internally.

For best results, use:

• 16 kHz sampling rate
• Mono audio
• Clear speech
• Minimal background noise
• Minimal reverberation

The Hugging Face processor handles the required feature extraction.

Supported Use Cases

This model can be used for:

• Persian speech recognition
• Farsi speech-to-text
• Persian audio transcription
• Voice assistants
• Conversational AI
• Voice-to-voice systems
• Customer support applications
• Persian NLP pipelines
• Speech analytics
• Meeting transcription
• Audio search
• Research and experimentation

Model Architecture

The model is based on the Whisper Large V3 Turbo architecture.

Whisper is an encoder-decoder Transformer architecture designed for multilingual speech recognition and related speech-processing tasks.

This model has been fine-tuned for improved performance on Persian/Farsi speech.

Fine-Tuning

Base Model

openai/whisper-large-v3-turbo

Target Language

Persian / Farsi

Language Code

fa

The fine-tuning process adapts the pretrained Whisper model to Persian speech characteristics, vocabulary, pronunciation patterns, and linguistic structure.

Evaluation

Recommended evaluation metrics for Persian ASR include:

• Word Error Rate (WER)
• Character Error Rate (CER)
• Normalized WER
• Normalized CER

For Persian ASR, normalization is particularly important because Unicode variants, Arabic/Persian character differences, whitespace, punctuation, and zero-width non-joiner usage can significantly affect raw WER and CER.

When reporting performance, evaluation should preferably be performed using a held-out test set representative of the intended deployment domain.

Limitations

ASR performance depends on the characteristics of the input audio and the target domain.

Potential sources of transcription errors include:

• Background noise
• Low-quality recordings
• Strong reverberation
• Regional accents and dialects
• Very fast speech
• Multiple speakers
• Overlapping speech
• Domain-specific terminology
• Uncommon names and words
• Code-switching between Persian and other languages

The model should be evaluated on representative production data before deployment in safety-critical or high-impact applications.

Production Considerations

For production deployment, consider:

• GPU inference for lower latency
• FP16 inference on compatible NVIDIA GPUs
• Audio preprocessing and normalization
• Voice Activity Detection (VAD)
• Chunking long audio files
• Batch inference for higher throughput
• Appropriate timeout and memory limits
• Domain-specific evaluation
• WER/CER monitoring
• Post-processing and Persian text normalization

For real-time applications, streaming audio should be segmented into appropriate chunks and processed incrementally rather than waiting for the complete recording.

Transformers Example

The model can be loaded directly through the standard Transformers APIs:

from transformers import (
    AutoModelForSpeechSeq2Seq,
    AutoProcessor,
)

model_id = "sadrasa97/whisper-large-v3-turbo-fa"

processor = AutoProcessor.from_pretrained(
    model_id
)

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id
)

Base Model

This model is based on:

openai/whisper-large-v3-turbo

Original base model:

https://huggingface.co/openai/whisper-large-v3-turbo

Citation

If you use this model in academic research, please cite the original Whisper publication:

@misc{radford2022robust,
    title={Robust Speech Recognition via Large-Scale Weak Supervision},
    author={
        Alec Radford and
        Jong Wook Kim and
        Tao Xu and
        Greg Brockman and
        Christine McLeavey and
        Ilya Sutskever
    },
    year={2022},
    eprint={2212.04356},
    archivePrefix={arXiv},
    primaryClass={eess.AS}
}

License

This model is based on openai/whisper-large-v3-turbo.

Please review the license and usage terms of the original base model, as well as the licenses and terms associated with any datasets used during fine-tuning.

Author

Sadra Saremi

Hugging Face:

https://huggingface.co/sadrasa97

Model Repository

https://huggingface.co/sadrasa97/whisper-large-v3-turbo-fa