Downloads · 30 days
0
trsdn/parakeet-tdt-0.6b-v3-htp-int8-16s
parakeet-tdt-0.6b-v3-htp-int8-16s is a automatic speech recognition model from trsdn. Use it when you need speech turned into text. It is set up for onnx. The card lists the license as cc-by-4.0.
Pre-compiled QAIRT context binary of the NVIDIA Parakeet TDT 0.6B v3 encoder for Qualcomm Hexagon NPU (HTP). Targets Snapdragon X Elite; inputs are pinned to a static 16-second audio window.
Downloads · 30 days
0
Access
Public
Updated Jun 3, 2026
Repo size
1.3 GB
Likes
0
Public
Click a slice to open those files.
.bin648 MB · 100%
From the Hugging Face model README
Pre-compiled QAIRT context binary of the NVIDIA Parakeet TDT 0.6B v3 encoder for Qualcomm Hexagon NPU (HTP). Targets Snapdragon X Elite; inputs are pinned to a static 16-second audio window.
This is the encoder only. The TDT decoder, mel preprocessor, vocab and config remain dynamic-shape and are sourced from the upstream ONNX export at istupakov/parakeet-tdt-0.6b-v3-onnx.
| File | Size | Purpose |
|---|---|---|
encoder-model.bin | 631 MB | QAIRT context binary (compiled by Qualcomm AI Hub) |
encoder-model.onnx | 408 B | EPContext wrapper — points ORT's QNN EP at the .bin |
nvidia/parakeet-tdt-0.6b-v3. The encoder uses a dynamic-shape
attention mask (Shape → Gather → Range → Expand) that the HTP backend
cannot evaluate. The mask was constant-folded against the fixed 8 s
window and the input shapes were frozen to [1, 128, 1601] /
[1]. See scripts/build_npu_encoder.py
in OpenWritr.submit_quantize_job with 32 FLEURS
calibration samples (weights_dtype=INT8, activations_dtype=INT16).
The INT8/INT16 mix is the standard HTP recipe for transformer encoders.submit_compile_job, target
Snapdragon X Elite CRD, options
--target_runtime qnn_context_binary --truncate_64bit_io.import os, numpy as np, onnxruntime as ort, onnxruntime_qnn as qep
os.add_dll_directory(qep.LIB_DIR_FULL_PATH)
ort.register_execution_provider_library("QNNExecutionProvider", qep.get_library_path())
npu_devs = [d for d in ort.get_ep_devices()
if d.device.type == ort.OrtHardwareDeviceType.NPU
and d.ep_name == "QNNExecutionProvider"]
so = ort.SessionOptions()
so.add_provider_for_devices(npu_devs, {"backend_type": "htp", "htp_performance_mode": "burst"})
sess = ort.InferenceSession("encoder-model.onnx", sess_options=so)
# audio_signal: float32 [1, 128, 1601] (mel features, 8 s of 16 kHz audio)
# length: int32 [1] (= 801)
out0, out1 = sess.run(None, {"audio_signal": mel, "length": np.array([1601], np.int32)})
# out0: float32 [1, 1024, 201] encoder features
# out1: int32 [1] encoded length
ort 2.0-rc.12's session builders crash inside QnnHtp when consuming an
EPContext-wrapper ONNX. See OpenWritr's
src/asr/qnn_ffi.rs
for a direct ort_sys FFI workaround that loads + runs cleanly in
native Rust apps.
The QNN backend (onnxruntime_providers_qnn.dll) loads several sibling
DLLs by name from os.add_dll_directory-registered paths:
QnnHtp.dll QnnHtpPrepare.dll QnnSystem.dll
QnnHtpV73Stub.dll libQnnHtpV73Skel.so libqnnhtpv73.cat
Without libQnnHtpV73Skel.so + libqnnhtpv73.cat, the V73 stub fails
LoadLibrary with ERROR_MOD_NOT_FOUND (126) and the HTP session
creation later aborts with STATUS_STACK_BUFFER_OVERRUN (0xC0000409).
Easy to miss when bundling for a fresh machine; you'll want all of them
alongside the EP DLL.
| Audio length | Decode (Preproc + NPU encode + TDT decoder) | × Realtime |
|---|---|---|
| 3 s | 128 ms (single window) | 23× |
| 5.8 s | 221 ms | 26× |
| 16.4 s | 375 ms (chunked, 3 windows w/ 1 s overlap) | 44× |
| 23.0 s | 626 ms (chunked, 4 windows) | 37× |
For comparison, CPU INT8 on the same chip runs the full pipeline at ~25× realtime average.
parakeet-tdt-0.6b-v3. Original
attribution preserved; please credit NVIDIA in derivative work.