sherpa-onnx-ort-zipformer-streaming-robust-es-v1

Spanish Zipformer Streaming Robust ES v1 — ONNX Runtime

This repository contains the ONNX Runtime (ORT) package for the Spanish streaming Zipformer v1 model. It recognizes whitespace-separated IPA phone symbols, not words. The package was exported from the exact evaluated average of epochs 76–80 in bookbot/zipformer-streaming-robust-es-v1:

• checkpoint: evaluated-avg-76-80.pt
• checkpoint SHA-256: d5cde97d82c214da39c3d9decffa84eaf6b15869c7b6521d2f184a2e59ad30e5
• causal streaming: chunk size 16, left context 128 frames
• feature input: 80-dimensional Kaldi fbank, 16 kHz, no dithering, snip_edges=false
• model architecture: the v1 six-stack atomic-phone Zipformer configuration in export-config.json

The six published models are fixed-optimization ORT files. The intermediate ONNX files and the epoch-999.pt scratch alias are deliberately not included.

Files

Usage

Install a compatible sherpa-onnx Python package, then run the included example from this directory:

python run_streaming.py

To use dynamic INT8 instead:

python run_streaming.py \
  --encoder encoder-evaluated-avg-76-80-chunk-16-left-128.int8.ort \
  --decoder decoder-evaluated-avg-76-80-chunk-16-left-128.int8.ort \
  --joiner joiner-evaluated-avg-76-80-chunk-16-left-128.int8.ort

The example uses CPU greedy streaming decoding, adds 0.3 seconds of tail padding, prints the phone-token sequence as JSON, validates every emitted token against tokens.txt, and reports real-time factor (RTF). For a different waveform, pass --audio path/to/audio.wav; it must be mono 16-bit PCM at 16 kHz.

Reproducing the export

Run these commands from icefall/egs/bookbot_es/ASR with ONNX Runtime 1.23.2. The exporter reads the exact materialized average, not a newly computed or single-epoch checkpoint:

native=tmp/zipformer-streaming-robust-es-v1
git lfs install
mkdir -p tmp
if [ ! -d "$native" ]; then
  git clone \
    https://huggingface.co/bookbot/zipformer-streaming-robust-es-v1 \
    "$native"
fi
scratch="$(mktemp -d)"
ln -s "$(realpath "$native/evaluated-avg-76-80.pt")" \
  "$scratch/epoch-999.pt"

python zipformer/export-onnx-streaming.py \
  --tokens "$native/data/lang_phone/tokens.txt" \
  --epoch 999 \
  --avg 1 \
  --use-averaged-model false \
  --exp-dir "$scratch" \
  --num-encoder-layers 2,2,2,2,2,2 \
  --downsampling-factor 1,2,4,8,4,2 \
  --feedforward-dim 512,768,768,768,768,768 \
  --num-heads 4,4,4,8,4,4 \
  --encoder-dim 192,256,256,256,256,256 \
  --query-head-dim 32 \
  --value-head-dim 12 \
  --pos-head-dim 4 \
  --pos-dim 48 \
  --encoder-unmasked-dim 192,192,192,192,192,192 \
  --cnn-module-kernel 31,31,15,15,15,31 \
  --decoder-dim 512 \
  --joiner-dim 512 \
  --context-size 2 \
  --causal true \
  --chunk-size 16 \
  --left-context-frames 128 \
  --use-transducer true

for component in encoder decoder joiner; do
  mv "$scratch/${component}-epoch-999-avg-1-chunk-16-left-128.onnx" \
    "$scratch/${component}-evaluated-avg-76-80-chunk-16-left-128.onnx"
  mv "$scratch/${component}-epoch-999-avg-1-chunk-16-left-128.int8.onnx" \
    "$scratch/${component}-evaluated-avg-76-80-chunk-16-left-128.int8.onnx"
done

export-onnx-streaming.py emits FP32 and dynamic-QInt8 ONNX files using opset 13. Convert all six to fixed-optimization ORT:

(
  cd "$scratch"
  python -m onnxruntime.tools.convert_onnx_models_to_ort \
    --optimization_style=Fixed \
    .
)

python -c 'import glob, sys, onnxruntime as ort; [ort.InferenceSession(p, providers=["CPUExecutionProvider"]) for p in glob.glob(sys.argv[1] + "/*.ort")]' "$scratch"
rm -f "$scratch"/*.onnx "$scratch"/epoch-999.pt

The converter also writes required_operators.config. Keep that file with the six ORT models. The intermediate ONNX files and epoch-999.pt alias are scratch-only and are intentionally absent from this repository. Full architecture, quantization operator, and output-name metadata is in export-config.json.

Data and licensing

The model was trained on the following sources:

• Common Voice 23.0 Spanish, source license CC0-1.0, with the versioned recipe cut publication at bookbot/common-voice-23-0-es-ipa-v1. Mozilla Data Collective source-use notices and attribution still apply.
• OpenSLR72, source license CC BY-SA 4.0, with the versioned recipe cut publication at bookbot/slr72-es-ipa-v1. Preserve the OpenSLR72 attribution and citation when redistributing derived data.
• TinyVox, used during training and evaluation under CC BY-NC-SA 3.0. TinyVox is child-speech data; this repository contains no TinyVox audio, metadata, rows, or utterance-level outputs. Only aggregate TinyVox metrics are reported below. The resulting weights and this ORT package are therefore released under the conservative CC BY-NC-SA 3.0 license and are non-commercial.

The included smoke waveform is the Common Voice test cut common_voice_es_19696062-9131, source recording common_voice_es_19696062, and is CC0-1.0. Its pinned source identity, transcript, resampling, and SHA-256 are recorded in sample-provenance.json.

Evaluation

The metric in this table is phoneme error rate (PER). Icefall's historical output filenames sometimes use the generic WER label (wer-summary-* and errs-*); those filenames do not change the metric, which is PER here.

The recommended decoder is modified beam search with num_active_paths=4. Common Voice 23.0 Spanish and SLR72 values reuse the recipe's official test splits for validation, so they are not independent holdout measurements. TinyVox test is the independently held-out result. TinyVox values are aggregate-only and do not expose utterance-level records.

Verification and hashes

Local validation loaded all six ORT files with ONNX Runtime 1.23.2 and ran CPU sherpa-onnx 1.13.6 streaming inference on the included waveform. FP32 emitted exactly the same 62-phone sequence as the canonical TorchScript inference from the checkpoint above. Dynamic INT8 completed successfully and emitted only tokens from the v1 inventory. The measured local RTFs and complete JSON output are in runtime-checks.json.

Core artifact SHA-256 values:

SHA256SUMS is authoritative for hashes of the configuration, provenance, runtime evidence, example, and this card. It does not include itself.