s1-mini-onnx

s1-mini ONNX (INT4)

This repository contains an unofficial ONNX INT4 conversion of superwhisper/s1-mini for use with ONNX Runtime GenAI and the ElBruno.LocalLLMs library.

This conversion is not endorsed by or affiliated with Superwhisper. All credit for the underlying model goes to the Superwhisper team; see their model card and LICENSE (included in this repository as LICENSE) for the original terms, including the naming clause that accompanies the Apache-2.0 license.

Model Description

s1-mini is a 596M-parameter (0.6B) speech-to-text transcript normalizer fine-tuned from Qwen/Qwen3-0.6B (Qwen3ForCausalLM, 28 layers, GQA 16 Q heads / 8 KV heads, head_dim 128, hidden_size 1024, vocab_size 151936, tied embeddings). It is not a general-purpose chat model — it performs a single task: cleaning up raw, lowercase, unpunctuated ASR transcripts into normalized text according to a control line.

Required Prompt Format

System prompt (verbatim — required):

You are a text normalizer for speech-to-text transcripts. The input begins with a control line specifying the styling, structure, and context settings; clean the transcript to match those settings and output only the cleaned text.

User message format — a control line followed by the raw transcript:

[Styling: semi-formal] [Structure: prose] [Context: general]
<raw lowercase unpunctuated asr transcript>

Supported control-line values include (non-exhaustive): Styling: semi-formal|formal|casual, Structure: prose|lists, Context: general|email|.... The model outputs only the cleaned transcript — no explanation, no preamble.

Decoding

Use enable_thinking=False and greedy decoding (do_sample=False, temperature=0, max_new_tokens=1024). Sampling is not validated for this model and may degrade output quality/consistency.

Note on enable_thinking=False: the model's chat template signals this by emitting an empty `<think>

</think>

block immediately after the<|im_start|>assistant `

header. If you build prompts manually instead of through the chat template, include that block explicitly.

Note on temperature: with do_sample=False, omit/avoid setting temperature=0.0 explicitly in onnxruntime-genai's GeneratorParams.set_search_options — some onnxruntime-genai builds divide logits by temperature even in greedy mode, which crashes on a literal 0.0. Leave temperature unset for greedy decoding.

Known Issues

• FP16 is currently broken on the CPU execution provider with onnxruntime-genai 0.15.1. Running the fp16/ variant fails with a shape-mismatch error inside the ONNX Runtime buffer-reuse optimizer, in the GQA repeat_kv Reshape node (InsertedPrecisionFreeCast_/model/layers.*/attn/v_proj/repeat_kv/Reshape_4). Use the int4/ variant for CPU inference until this is confirmed fixed in a newer onnxruntime-genai release. The fp16/ artifact is still published here for future compatibility/GPU experimentation, but it is not currently validated to run.
• Calling tokenizer.decode() on an empty token sequence (which can legitimately happen — e.g. pure-filler input that should normalize to nothing) crashes the native onnxruntime-genai decoder with an integer divide-by-zero. Guard for a zero-length generated sequence in calling code and treat it as an empty string instead of calling decode().

Conversion Details

Usage with ElBruno.LocalLLMs

using var client = await LocalChatClient.CreateAsync(new LocalLLMsOptions
{
    Model = KnownModels.S1Mini,
    EnsureModelDownloaded = true   // downloads automatically on first run
});

var response = await client.CompleteAsync(
    "[Styling: semi-formal] [Structure: prose] [Context: general]\n" +
    "so um i need to like send the the report by uh friday no wait make that thursday");

License

This conversion is distributed under the same terms as the source model: Apache-2.0 with an additional naming clause — see the included LICENSE file or https://huggingface.co/superwhisper/s1-mini/raw/main/LICENSE for the authoritative text.