Downloads · 30 days
0
qualcomm-ai-hub-community/whisper-base-multilingual-qcs6490-qnn-int8
whisper-base-multilingual-qcs6490-qnn-int8 is a automatic speech recognition model from qualcomm-ai-hub-community. Use it when you need speech turned into text. The card lists the license as mit.
First published Whisper build compiled and profiled for the Qualcomm Dragonwing QCS6490 (Hexagon HTP), the SoC on Rubik Pi 3 / RB3 Gen 2 / Radxa Dragon Q6A class devices. Fills the gap discussed in qualcomm/ai-hub-mod…
Downloads · 30 days
0
Access
Public
Updated Aug 2, 2026
Repo size
103 MB
Likes
0
Public
Click a slice to open those files.
.bin103 MB · 100%
From the Hugging Face model README
First published Whisper build compiled and profiled for the Qualcomm Dragonwing QCS6490 (Hexagon HTP), the SoC on Rubik Pi 3 / RB3 Gen 2 / Radxa Dragon Q6A class devices. Fills the gap discussed in qualcomm/ai-hub-models#281.
| Component | Inference | Peak memory | Profile job |
|---|---|---|---|
| Encoder (30 s window, tail-trimmable) | 49.5 ms | 13.8 MB | j57966krg |
| Decoder (per token) | 12.1 ms | 26.4 MB | jgnkxxomg |
≈ 360 ms for a 30-token transcript — comfortably real-time speech recognition on a 12-TOPS IoT NPU.
Measured constraint map for this target (all jobs on AI Hub, 2026-08-02):
qnn_context_binary and precompiled_qnn_onnx. 16-bit activations appear unsupported on this HTP generation (jobs jgjq11k85, jgznwwv6g, jgnkxxxmg, jpymnnn4p).--quantize_full_type int8 --quantize_io) works — these binaries.No local AIMET is needed at any point (AIMET-ONNX is unavailable on macOS/aarch64 — the original blocker in #281): quantization here happens server-side at compile; for accuracy-critical deployments, use AI Hub's hosted quantize jobs with calibration audio, then compile the same way.
# Python >= 3.10; pip install "qai-hub-models[whisper-base]"; qai-hub configure --api_token <yours>
python -m qai_hub_models... # or drive qai_hub directly:
import qai_hub as hub
from qai_hub_models.models._shared.hf_whisper.model import HfWhisperEncoder, HfWhisperDecoder
device = hub.Device("Dragonwing RB3 Gen 2 Vision Kit")
for name, cls in (("encoder", HfWhisperEncoder), ("decoder", HfWhisperDecoder)):
component = cls.from_pretrained(hf_whisper_version="openai/whisper-base").eval()
spec = component.get_input_spec()
job = hub.submit_compile_job(
model=component.convert_to_torchscript(spec), device=device, input_specs=spec,
options="--target_runtime qnn_context_binary --quantize_full_type int8 --quantize_io",
name=f"whisper-base-{name}-int8")
Note: blind (calibration-free) INT8 trades some accuracy for availability — validate WER on your domain before production use. The same recipe works for any HF Whisper fine-tune (e.g., vinai/PhoWhisper-small for Vietnamese) by changing hf_whisper_version.
Quantized I/O per --quantize_io — exact tensor specs are on each job page above. Runs via Qualcomm AI Engine Direct (QNN) context-binary loading, or ONNX Runtime QNN EP with context binary support.
--quantize_full_type int8). WER against the FP32 baseline has not been measured — validate on your domain before production use; for accuracy-critical deployments run an AI Hub hosted quantize job with calibration audio first.Base model: openai/whisper-base (MIT). Compiled artifacts follow the base model's MIT license. Export wrappers: qualcomm/ai-hub-models (BSD-3-Clause, SHA-attention rewrite). Prepared by Phạm Văn Ngoan. Not an official Qualcomm release; community-contributed, moderated for structure only.