Downloads · 30 days
0
Noir-Lime/qwen3-forced-aligner-0.6b-onnx-int8
qwen3-forced-aligner-0.6b-onnx-int8 is a machine learning model from Noir-Lime. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for onnxruntime. The card lists the license as apache-2.0.
CPU inference graphs for word-level forced alignment, derived from Qwen/Qwen3-ForcedAligner-0.6B. Intended for ARM (Ampere Altra / Neoverse N1) and x86 servers where the fp32 model is too slow or too large; ONNX Runti…
Downloads · 30 days
0
Access
Public
Updated Sep 19, 2026
Repo size
1.1 GB
Likes
0
Public
Click a slice to open those files.
.onnx802 MB · 71%
From the Hugging Face model README
CPU inference graphs for word-level forced alignment, derived from
Qwen/Qwen3-ForcedAligner-0.6B.
Intended for ARM (Ampere Altra / Neoverse N1) and x86 servers where the fp32
model is too slow or too large; ONNX Runtime's MLAS int8 kernels use sdot on
ARM and AVX2/VNNI on x86.
| Base model | Qwen/Qwen3-ForcedAligner-0.6B, revision c7cbfc2048c462b0d63a45797104fc9db3ad62b7 (BF16) |
| Quantization | SmoothQuant fold (α = 0.5, scales in qwen3-forced-aligner-smoothquant.safetensors, 184 groups, 9 English calibration clips) → ONNX Runtime quantize_dynamic, per-tensor symmetric INT8 weights, dynamic u8 activations, every MatMul in the audio encoder and text decoder. Conv stack stays FP32. |
| Attention | eager (SDPA/GQA has no ONNX symbolic); prebuilt 4-D causal mask |
| Toolchain | torch 2.7.1+cpu · transformers 4.57.6 · onnx 1.23.0 · onnxruntime 1.30.0 · opset 17 |
| Export host | x86-64 (graphs are architecture-independent; MLAS selects kernels at run time) |
| File | Content | Size |
|---|---|---|
audio-conv.onnx | mel conv stack + conv_out, FP32; input padded_feature (chunks, 1, 128, 100) | 46 MB |
audio-encoder.onnx | 24 audio layers + ln_post/proj1/proj2, INT8; input hidden (frames, 1024) | 292 MB |
text-decoder.onnx | 28 text layers + lm_head + argmax, INT8; inputs inputs_embeds (1, L, 1024), position_ids (3, 1, L) | 427 MB |
embed-tokens.safetensors | BF16 token table embed_tokens + FP32 audio positional table positional_embedding | 303 MB |
tokenizer.json | Qwen2 BPE tokenizer (HF tokenizers format, exported with fix_mistral_regex=True) for runtimes without transformers | 11 MB |
aligner-runtime.json | timestampTokenId, timestampSegmentTime, audioTokenId, nWindow, convChunkSize, hiddenSize | 139 B |
qwen3-forced-aligner-smoothquant.safetensors | SmoothQuant scales used for this export (audit / regeneration) | 1 MB |
The graphs replace only thinker(**inputs).logits.argmax(-1) from
qwen_asr.Qwen3ForcedAligner.align; the mel/tokeniser processor
(Qwen3ASRProcessor) and the timestamp encoder/parser
(Qwen3ForceAlignProcessor) from qwen-asr==0.0.6 are reused unchanged.
Per clip:
processor(text=[aligner_text], audio=[wav], return_tensors="np") →
input_ids, input_features (128 × T mel), feature_attention_mask.audio-conv.onnx, add positional_embedding[:t], keep the first
out_len(chunk_len) frames of each chunk where
out_len(n) = (((n % 100 − 1) // 2 + 1 − 1) // 2 + 1 − 1) // 2 + 1 + (n // 100) · 13,
concatenate, run audio-encoder.onnx.embed_tokens[input_ids] → FP32, scatter the encoder output into the
positions where input_ids == audioTokenId, build position_ids as
arange(L) broadcast to (3, 1, L), run text-decoder.onnx.output_ids at positions where input_ids == timestampTokenId,
multiply by timestampSegmentTime (ms), pass to parse_timestamp.Requires the base snapshot's config.json, tokenizer_config.json and
preprocessor_config.json; model.safetensors is not needed. A reference
TypeScript implementation of steps 1–4 (Whisper log-mel with an exact 400-point
DFT, @huggingface/tokenizers, onnxruntime-node) reproduces the Python
processor's token ids exactly and mel features to 4e-5.
| fp32 torch | this export | |
|---|---|---|
| Inference | 48.6 s | 19.0 s |
| Steady RSS | 3.96 GiB | 1.68 GiB |
| Peak RSS | 6.14 GiB | 1.92 GiB |
| Timestamps vs fp32 | — | 10/231 words differ, all ≤ 1 frame (80 ms), p95 0, no invalid intervals |
Calibration audio was a single English speaker in a quiet room; accuracy on other speakers, noisy audio or other languages has not been measured. The scales file contains per-channel activation statistics only, no audio.