Downloads · 30 days
39
1% of all-time downloads
Flurin17/whisper-large-v3-peft-swiss-german
whisper-large-v3-peft-swiss-german is a machine learning model from Flurin17. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
As of our knowledge SOTA in swiss german with wer=14.269151618793657 and normalizedwer=12.800897299473698.
Downloads · 30 days
39
1% of all-time downloads
All-time downloads
6.3K
Public
Parameters
907M
1.1 GB on disk
Likes
7
Public
Click a slice to open those files.
.safetensors1 GB · 100%
How the weights are stored.
I8839M · 92%
From the Hugging Face model README
As of our knowledge SOTA in swiss german with wer=14.269151618793657 and normalized_wer=12.800897299473698.
Trained on RTX 3070 for 30 hours using SwissDial all Dialects with following guide: https://github.com/Vaibhavs10/fast-whisper-finetuning/blob/main/Whisper_w_PEFT.ipynb
model_name_or_path = "openai/whisper-large-v3"
task = "transcribe"
import json
import os
from transformers import WhisperFeatureExtractor
from transformers import WhisperTokenizer
feature_extractor = WhisperFeatureExtractor.from_pretrained(model_name_or_path)
tokenizer = WhisperTokenizer.from_pretrained(model_name_or_path, task=task)
from peft import PeftModel, PeftConfig
from transformers import WhisperForConditionalGeneration, Seq2SeqTrainer
peft_model_id = "flurin17/whisper-large-v3-peft-swiss-german" # Use the same model ID as before.
peft_config = PeftConfig.from_pretrained(peft_model_id)
model = WhisperForConditionalGeneration.from_pretrained(
peft_config.base_model_name_or_path, load_in_8bit=True, device_map="auto"
)
model = PeftModel.from_pretrained(model, peft_model_id)
model.config.use_cache = True
from transformers import AutomaticSpeechRecognitionPipeline
import torch
pipe = AutomaticSpeechRecognitionPipeline(model=model, tokenizer=tokenizer, feature_extractor=feature_extractor)
with torch.cuda.amp.autocast():
result = pipe(r"L:\random\audio.mp3", generate_kwargs={"language": "german"})
print(result["text"])