Downloads · 30 days
0
qualcomm-ai-hub-community/whisper-small-multilingual-qcs6490-qnn-int8
whisper-small-multilingual-qcs6490-qnn-int8 is a automatic speech recognition model from qualcomm-ai-hub-community. Use it when you need speech turned into text. The card lists the license as mit.
Companion to the whisper-base build — the higher-accuracy Whisper-Small variant compiled and profiled for the Qualcomm Dragonwing QCS6490 (Hexagon HTP), the SoC on Rubik Pi 3 / RB3 Gen 2 / Radxa Dragon Q6A class devic…
Downloads · 30 days
0
Access
Public
Updated Aug 2, 2026
Repo size
294 MB
Likes
1
Public
Click a slice to open those files.
.bin294 MB · 100%
From the Hugging Face model README
Companion to the whisper-base build — the higher-accuracy Whisper-Small variant compiled and profiled for the Qualcomm Dragonwing QCS6490 (Hexagon HTP), the SoC on Rubik Pi 3 / RB3 Gen 2 / Radxa Dragon Q6A class devices. Fills the gap discussed in qualcomm/ai-hub-models#281.
| Component | Inference | Peak memory | Profile job |
|---|---|---|---|
| Encoder (30 s window, tail-trimmable) | 188.7 ms | 32.7 MB | jp3688xzp |
| Decoder (per token) | 33.8 ms | 73.8 MB | jgznww86g |
≈ 1.0 s for a 30-token transcript; the smaller whisper-base build (also in this org) runs ≈ 3× faster where its accuracy suffices.
Measured constraint map for this target (all jobs on AI Hub, 2026-08-02):
qnn_context_binary and precompiled_qnn_onnx. 16-bit activations appear unsupported on this HTP generation (jobs jgjq11k85, jgznwwv6g, jgnkxxxmg, jpymnnn4p).--quantize_full_type int8 --quantize_io) works — these binaries.No local AIMET is needed at any point (AIMET-ONNX is unavailable on macOS/aarch64 — the original blocker in #281): quantization here happens server-side at compile; for accuracy-critical deployments, use AI Hub's hosted quantize jobs with calibration audio, then compile the same way.
# Python >= 3.10; pip install "qai-hub-models[whisper-small]"; qai-hub configure --api_token <yours>
python -m qai_hub_models... # or drive qai_hub directly:
import qai_hub as hub
from qai_hub_models.models._shared.hf_whisper.model import HfWhisperEncoder, HfWhisperDecoder
device = hub.Device("Dragonwing RB3 Gen 2 Vision Kit")
for name, cls in (("encoder", HfWhisperEncoder), ("decoder", HfWhisperDecoder)):
component = cls.from_pretrained(hf_whisper_version="openai/whisper-small").eval()
spec = component.get_input_spec()
job = hub.submit_compile_job(
model=component.convert_to_torchscript(spec), device=device, input_specs=spec,
options="--target_runtime qnn_context_binary --quantize_full_type int8 --quantize_io",
name=f"whisper-small-{name}-int8")
Note: blind (calibration-free) INT8 trades some accuracy for availability — validate WER on your domain before production use. The same recipe works for any HF Whisper fine-tune (e.g., vinai/PhoWhisper-small for Vietnamese) by changing hf_whisper_version.
Quantized I/O per --quantize_io — exact tensor specs are on each job page above. Runs via Qualcomm AI Engine Direct (QNN) context-binary loading, or ONNX Runtime QNN EP with context binary support.
--quantize_full_type int8). WER against the FP32 baseline has not been measured — validate on your domain before production use; for accuracy-critical deployments run an AI Hub hosted quantize job with calibration audio first.Base model: openai/whisper-small (MIT). Compiled artifacts follow the base model's MIT license. Export wrappers: qualcomm/ai-hub-models (BSD-3-Clause, SHA-attention rewrite). Prepared by Phạm Văn Ngoan. Not an official Qualcomm release; community-contributed, moderated for structure only.