Whisper-Large-v3-Turbo-OpenVINO-INT4

Celeste Imperia | Whisper-Large-v3-Turbo (OpenVINO INT4 Gold)

The specialized transcription engine for edge compute. This is a hardware-optimized port of OpenAI's Whisper-Large-v3-Turbo, forged using Intel's NNCF for ultra-low latency execution on CPUs, GPUs, and NPUs.

---

🐍 For Python Users (Optimum Intel)

To achieve the highest throughput on Intel hardware, use the optimum-intel library with the OpenVINO backend.

from optimum.intel import OVModelForSpeechSeq2Seq
from transformers import AutoTokenizer, AutoFeatureExtractor, pipeline

model_id = "CelesteImperia/Whisper-Large-v3-Turbo-OpenVINO-INT4"

# Load model with OpenVINO optimization
model = OVModelForSpeechSeq2Seq.from_pretrained(model_id, device="CPU") # or "GPU"/"NPU"
tokenizer = AutoTokenizer.from_pretrained(model_id)
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=tokenizer,
    feature_extractor=feature_extractor,
    device="cpu", # OpenVINO handles the device mapping
)

result = pipe("audio_sample.wav")
print(result["text"])

---

💻 For C# / .NET Users (OpenVINO .NET SDK)

As a Senior .NET developer, I have validated this model for native C# integration. This allows for high-performance transcription in Windows desktop applications without Python overhead.

using OpenVinoSharp;
using System;

// 1. Initialize OpenVINO Core
Core core = new Core();

// 2. Load the Optimized IR Model (.xml / .bin)
Model model = core.read_model("whisper_v3_turbo_int4.xml");
CompiledModel compiledModel = core.compile_model(model, "CPU");

// 3. Create Infer Request
InferRequest request = compiledModel.create_infer_request();

Console.WriteLine("🚀 Whisper OpenVINO Engine Initialized on .NET");
// [Insert preprocessing and inference logic here]

---

📜 Gold Release: Change Log

This release marks the transition from FP16 weights to a fully quantized INT4 NNCF architecture, specifically tuned for the Whisper-Turbo decoder.

🛠️ Key Technical Upgrades:

• Weight Compression: Reduced model footprint from ~1.6GB to ~450MB using Intel NNCF Asymmetric Quantization.
• Architecture: Optimized the "Turbo" decoder blocks, which feature only 4 layers compared to the original Large-v3's 32 layers.
• NPU Ready: Validated for Intel Core Ultra (Meteor Lake) NPU execution, enabling background transcription with near-zero CPU impact.
• Mixed Precision: Utilizes INT4/INT8 Mixed Precision to maintain Word Error Rate (WER) parity with the original FP16 model while gaining 3x inference speed.

---

🎯 Sample Performance Benchmarks

Test Case 1: Short Clip (30s Audio)

Result: Transcription finished in ~1.2 seconds on i5-11400 (CPU Only). Accuracy: Perfect alignment with zero "hallucination" in the silence padding.

Test Case 2: Technical Terminology

Context: Audio containing complex technical jargon (e.g., "Quantization," "NPU-Native," "CASA Ratios"). Accuracy: The model maintains high fidelity for specialized domains, outperforming standard small/medium Whisper models.

---

🏗️ Technical Forge & Infrastructure

• Model Type: OpenVINO IR (INT4 Quantized)
• Optimization Tooling: Intel Neural Network Compression Framework (NNCF)
• Training Workstation: Dual-GPU (NVIDIA RTX 3090 24GB + RTX A4000 16GB)
• Memory: 64GB DDR4

---

📜 License & Disclaimer

License: This project is licensed under the Apache License 2.0, aligning with the original Whisper-Turbo release by OpenAI.

Disclaimer: This model is provided for research and accessibility purposes. While optimized for accuracy, users should verify critical transcriptions. Celeste Imperia is not liable for errors in high-stakes environments (Legal/Medical).

---

☕ Support the Forge

Maintaining an AI workstation and hosting hardware-optimized models requires significant compute resources. If these tools power your workflow, consider supporting our development:

Scan to support via UPI (India Only):

<img src="https://huggingface.co/datasets/CelesteImperia/Assets/resolve/main/QrCode.jpeg" width="200">

---

Connect with the architect: Abhishek Jaiswal on LinkedIn