whisper-small-ne-en-finetuned-v2

whisper-small-ne-en-finetuned-v2

This model is a fine-tuned version of openai/whisper-small for Nepali and English code-switched speech recognition.

Model Details

• Base Model: openai/whisper-small
• Language: Nepali-English (code-switched)
• Task: transcribe
• Fine-tuned on: Custom Nepali ASR dataset

Training Configuration

• Epochs: 3
• Batch Size: 8 x 2 = 16 (effective)
• Learning Rate: 1e-05
• Mixed Precision: FP16
• Training Samples: 7576
• Validation Samples: 945
• Test Samples: 948

Evaluation Results

Usage

from transformers import WhisperProcessor, WhisperForConditionalGeneration
import librosa

# Load model and processor
processor = WhisperProcessor.from_pretrained("Bijay13/whisper-small-ne-en-finetuned-v2")
model = WhisperForConditionalGeneration.from_pretrained("Bijay13/whisper-small-ne-en-finetuned-v2")

# Load audio
audio, sr = librosa.load("your_audio.wav", sr=16000)

# Transcribe
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features
predicted_ids = model.generate(input_features)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

License

This model is released under the Apache 2.0 license.