whisper-medium-ternary

Whisper-medium — Ternary Quantized (tritplane3)

Ternary-quantized version of openai/whisper-medium.

Specifications

Usage

from ternary_quant.inference import load_ternary_model
import torch, numpy as np

model, proc = load_ternary_model("AsadIsmail/whisper-medium-ternary", runtime_mode="cached", device="cpu")
model = model.float()  # Required for encoder compat

# Transcribe audio
import soundfile as sf
audio, sr = sf.read("audio.flac")
inputs = proc(audio.astype(np.float32), sampling_rate=sr, return_tensors="pt")
inputs = {k: v.float() for k, v in inputs.items()}
with torch.no_grad():
    ids = model.generate(**inputs, max_new_tokens=100)
print(proc.batch_decode(ids, skip_special_tokens=True)[0])

Collection

Part of ternary-models.