Downloads · 30 days
34
16% of all-time downloads
diarizeapp/granite-speech-5.0-470m-turboctc-onnx
granite-speech-5.0-470m-turboctc-onnx is a automatic speech recognition model from diarizeapp. Use it when you need speech turned into text. The card lists the license as apache-2.0.
This repository provides ONNX Runtime weights for ibm-granite/granite-speech-5.0-470m-turboctc, optimized for low-latency real-time speech recognition in diarize.
Downloads · 30 days
34
16% of all-time downloads
All-time downloads
218
Public
Repo size
11.5 GB
Likes
1
Public
Click a slice to open those files.
.onnx4.3 GB · 100%
From the Hugging Face model README
This repository provides ONNX Runtime weights for ibm-granite/granite-speech-5.0-470m-turboctc, optimized for low-latency real-time speech recognition in diarize.
model.onnx: Full Conformer CTC graph with dynamic axes [batch_size, time, 320] -> [batch_size, time, 16384]. Works on any hardware via ONNX Runtime CPU / DirectML / CUDA EPs.model_npu_static.onnx: Static-shape variant [1, 200, 320] (200 frames = 1.6 s streaming window) for compiling a Hexagon NPU EPContext on ARM64 / Snapdragon X Elite via scripts/compile_qnn_granite.py.model_quantized.onnx: Dynamic INT8 (W8A32) quantized variant (QInt8 weights, FP32 activations, CTC head preserved to ensure exact logit margins and zero blank collapse) for accelerated CPU / mobile / edge inference.tokenizer.json, config.json, preprocessor_config.json, processor_config.json, generation_config.json, tokenizer_config.json: tokenizer and configuration files.model_quantized.onnx is produced with ONNX Runtime's dynamic quantization, quantizing Conformer MatMul weights to signed INT8 while preserving full float32 activations and excluding the final CTC projection head. This ensures 100% transcript accuracy with 0 blank collapse and a 70%+ reduction in memory footprint:
uv run python scripts/export_granite_speech_onnx.py --trust-remote-code --quantize dynamic
The QNN execution provider (onnxruntime-qnn 2.x) is a plugin EP: register its library with ONNX Runtime and attach it with SessionOptions.add_provider_for_devices. scripts/compile_qnn_granite.py does this, verifies that the provider is really active, and writes the EPContext wrapper plus context binary next to the model. Check your setup first with:
uv run python scripts/compile_qnn_granite.py --diagnose
uv run python scripts/compile_qnn_granite.py --model-dir /path/to/this/repo
Note: The compiled context binary is device specific and is not part of this repository. Each machine compiles its own from the static or quantized model.
Apache-2.0 (preserved from the upstream IBM Granite release).