moonshine-tiny-uk-onnx

Moonshine Tiny Ukrainian — ONNX export

ONNX graphs for moonshine-ai/moonshine-tiny-uk, exported for on-device speech recognition through ONNX Runtime. Weights, training data and every design decision in the model belong to Moonshine AI — this repository only converts what they published, and is redistributed under their licence.

Powered by Moonshine AI.

Source

• Model: moonshine-ai/moonshine-tiny-uk — 27.1M parameters, Ukrainian
• Authors: Moonshine AI (formerly Useful Sensors)
• Reported by them for Ukrainian: 18.25 WER on Fleurs, 26.11 on Common Voice 17 — against Whisper tiny's 63.83 / 67.07 and Whisper medium's 11.62 / 20.9, at a fraction of the size

Please cite and support the original work.

Why ONNX, and why this shape

Moonshine takes raw 16 kHz audio and does not pad it to a fixed window, so the cost of a two-second utterance is the cost of two seconds. That is the whole reason to prefer it on a phone over a Whisper-family model, whose encoder always processes its 30-second window and therefore charges the same for a short command as for a long sentence.

Three graphs, so the decoder's KV cache survives between tokens:

The int8 set is 45.8 MB in total.

Inputs / outputs

encoder.onnx
  input_values          float32 [batch, samples]        raw waveform, 16 kHz, mono
  → encoder_hidden_states float32 [batch, frames, 288]  ~41 frames per second

decoder_init.onnx
  input_ids             int64  [batch, 1]               decoder_start_token_id (1)
  encoder_hidden_states float32 [batch, frames, 288]
  → logits              float32 [batch, 1, 32768]
  → present_{key,value}_self_{0..5}    [batch, 8, 1, 36]
  → present_{key,value}_cross_{0..5}   [batch, 8, frames, 36]

decoder_step.onnx
  input_ids             int64  [batch, 1]               previous token
  cache_position        int64  [1]                      absolute index of this token
  encoder_hidden_states float32 [batch, frames, 288]
  past_{key,value}_self_{0..5}, past_{key,value}_cross_{0..5}
  → logits, present_{key,value}_self_{0..5}

Greedy decoding: run the encoder once, decoder_init once, then decoder_step until the argmax is the EOS token (2). Feed back the self-attention KV each step; the cross-attention KV never changes for an utterance.

Three things that will cost you an evening if nobody says them

1. decoder_step needs encoder_hidden_states even though the cross-attention KV is already in the cache. Without it the decoder silently skips cross-attention and produces fluent, confident, entirely invented text. It does not error.
2. cache_position is the absolute index of the current token — 1 for the first generated one, not 0.
3. The export needs attn_implementation="eager". With the default SDPA path, torch.onnx.export fails on enable_gqa=True where query and key heads are equal.

What was verified

Against the PyTorch model, on this exact export:

• encoder parity, fp32: max|torch − onnx| = 8.7e-06
• cached decoding returns the same text as cache-free decoding on every sample tried
• three Fleurs uk_ua utterances: two transcribed word-for-word against the reference, the third with one wrong word that PyTorch gets wrong too — so the conversion is not what lost it
• int8 keeps the text intact apart from a single word across the three samples
• encoder: 0.15–0.3 s for 7–9.5 s of audio; cached decode 15–32 ms per token (fp32, x86 CPU in Docker). The cache is 2–4× faster than re-running the full sequence per token.

Reproducing

export.py, export_cached.py, quantize.py, quantize_cached.py and validate_cached.py are included. torch 2.8, transformers 5.0, onnx, onnxscript, onnxruntime.

License

Moonshine AI Community License Agreement — see LICENSE.txt, copied from the source repository. It permits commercial use below USD 1,000,000 of annual revenue, requires registration above it, and requires that materials built on the model display “Powered by Moonshine AI”. The rights to the model are Moonshine AI's; this export changes nothing about that.