hohe-asr-amharic-onnx-int8

Hohe ASR Amharic — ONNX INT8

CPU-oriented ONNX INT8 quantization of snapwre/hohe-asr-amharic, an Amharic CTC speech-recognition model.

This repository contains a derived model, not a new training run. Please cite and credit the original model and its creators when using or redistributing it.

What changed

• Exported the original Wav2Vec2BertForCTC checkpoint to ONNX.
• Dynamically quantized MatMul and Gemm weights with ONNX Runtime.
• Used per-channel signed INT8 weights with reduced range for AVX2 CPUs without VNNI. Convolution layers remain FP32 because ONNX Runtime's CPU provider does not supply the required 1-D ConvInteger kernel.
• Kept the same tokenizer, 16 kHz frontend, 416-token vocabulary, and CTC decoding behavior as the source model.

Complete benchmark results

Measured locally on an Intel Core i5-8350U using 4 ONNX Runtime threads.

Speed is seconds of audio processed per second of compute; values above 1.00x are faster than real time.

The published INT8 graph reached 97.6% frame-level argmax agreement with the PyTorch source model on identical random inputs, with mean posterior KL divergence 0.08281. The NumPy frontend matched Hugging Face's SeamlessM4TFeatureExtractor to a maximum absolute difference of 9.537e-7 for raw filterbank features and 1.073e-5 for model inputs.

Accuracy method

• Dataset: Google FLEURS am_et test split.
• Evaluation subset: 24 clips / 5.8 minutes, selected shortest-first within an audio budget.
• Scoring: corpus CER/WER after Ge'ez homophone normalization, language-tag removal, and punctuation removal.
• The original model card's full 516-clip greedy result is 6.28% CER / 17.86% WER. That result belongs to the source model; it is shown only as context.

Speed method and uncertainty

• Hardware: Intel Core i5-8350U, 4 cores / 8 threads, AVX2 without VNNI.
• Runtime: ONNX Runtime CPU execution provider with 4 intra-op threads.
• Every tier was measured in rotating rounds to reduce ordering bias.
• The table reports the best observed round. Thermal throttling caused up to a 3.75x spread between rounds, so these are machine-specific measurements, not universal performance promises.
• FP32 at 22-second chunks paged heavily on the 16 GB test machine and was intentionally omitted instead of reporting a misleading result.

The small, non-random accuracy subset must not be treated as a replacement for the source model's full evaluation or as a statistically representative estimate. Results on other CPUs, runtimes, chunk lengths, and audio domains will differ.

Interactive benchmark report and raw measurements

The complete local Performance page is included in benchmark-site/. Download that directory and open performance.html to explore every tier, chunk length, accuracy comparison, tuning knob, and minimum/recommended/best hardware note. It has no CDN or network dependency.

Machine-readable evidence is included under benchmarks/:

• benchmarks.json — all displayed benchmark values, methodology, hardware guidance, accuracy comparisons, and parity summary.
• parity.json — ONNX-vs-PyTorch and frontend parity measurements.
• speed.log — the underlying multi-round timing output, including thermal variation and tier ordering.

Only the recommended per-channel reduced-range INT8 weights are published in this repository. Rows for the other tiers document the selection experiment; they do not imply that those model files are present here.

Files

• model.onnx — dynamically quantized acoustic model.
• assets/frontend.npz — frontend constants used by the accompanying local Hohe STT implementation.
• assets/meta.json — vocabulary IDs and model/frontend metadata.
• Hugging Face tokenizer and preprocessor configuration files are included at the repository root.
• benchmark-site/ — self-contained detailed benchmark viewer.
• benchmarks/ — machine-readable benchmark and parity evidence plus raw timing output.

The model expects 16 kHz mono audio converted to the same 160-bin frontend features used by the original Wav2Vec2-BERT processor. Output is CTC logits; strip the initial [AMH] language token after decoding.

Limitations

The original model is Amharic-only and does not add punctuation or casing. It is weaker on spontaneous conversation, overlapping speech, and languages other than Amharic. Quantization can change some predictions; use the original FP32 checkpoint when maximum fidelity is required.

Attribution

• Original model: snapwre/hohe-asr-amharic
• Original model creators: Chapi and the Dataset.ET contributors
• Upstream base model: badrex/Ethio-ASR-multilingual-600M
• Quantization and ONNX packaging: Yaredoffice
• Public evaluation data: Google FLEURS, Amharic am_et
• Quantization tooling: ONNX Runtime

License

CC BY 4.0, inherited from the source model. You may use and redistribute this model, including commercially, provided that appropriate attribution is given. The source model's training corpus is not distributed here and is not covered by this repository's model license.