whisper-large-v3-4bit

Model Card for Model ID

This model is a quantized version of openai/whisper-large-v3, optimized for more efficient use while maintaining performance.

Model Details

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: alicekyting (based on OpenAI's Whisper model)
• Model type: Speech recognition model
• Language(s) (NLP): Multilingual

Uses

This model can be used for automatic speech recognition (ASR) tasks, including transcription and translation. It's particularly useful in scenarios where computational efficiency is important, as it has been quantized to 4-bit precision.

Hardware Requirements

It is recommended to use this model on a device with a compatible GPU.

Bias, Risks, and Limitations

This model inherits any biases, risks, and limitations present in the original openai/whisper-large-v3 model. Additionally, the quantization process may introduce slight degradation in accuracy compared to the original model.

Recommendations

Users should be aware of the trade-off between efficiency and potential minor accuracy loss due to quantization. It's recommended to evaluate the model's performance on your specific use case before deployment.

How to Get Started with the Model

Use the following code to load and use the model:

from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
import torch

# Load the model
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    "alicekyting/whisper-large-v3-4bit",
    device_map="auto",
    torch_dtype=torch.float16,
    use_safetensors=True,
)

# Load the processor
processor = AutoProcessor.from_pretrained("alicekyting/whisper-large-v3-4bit")

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    torch_dtype=torch.float16,
    device_map="auto"
)