whisper-hindi2hinglish-swift-ONNX

Whisper-Hindi2Hinglish-Swift (ONNX, for transformers.js)

This is an ONNX conversion of Oriserve/Whisper-Hindi2Hinglish-Swift (whisper-base-architecture, fine-tuned for code-switched Hindi-English ("Hinglish") speech recognition), packaged for use with 🤗 Transformers.js.

All credit for the underlying model goes to Oriserve. See the base model's card for training data, benchmarks, and full details. License is inherited as Apache-2.0 from the base model.

Output format

This model outputs romanized (Latin-script) text for Hindi portions of speech (e.g. "vah bas din mein kitni baar chalti hai?"), not Devanagari script - this matches the base model's design and how Hinglish is commonly written in practice.

Files

• onnx/encoder_model_fp16.onnx - encoder, fp16
• onnx/decoder_model_merged.onnx - merged decoder (with/without KV-cache branches), fp32

The decoder is kept in fp32 rather than fp16: fp16 conversion of the merged conditional (If-branching) decoder graph repeatedly hit ONNX Runtime type-inference errors in nested subgraphs (stray Cast nodes and KV-cache output types not fully updated by the standard float16 conversion tooling). Rather than keep patching individual nodes, the decoder was left in fp32 - the encoder's fp16 conversion alone already keeps the total well under typical browser-deployment size budgets. Recommended dtype config when loading via transformers.js:

const transcriber = await pipeline("automatic-speech-recognition", "<this-repo-id>", {
  dtype: { encoder_model: "fp16", decoder_model_merged: "fp32" },
});

Total download size: ~345MB.

Conversion process

1. optimum-cli export onnx --model Oriserve/Whisper-Hindi2Hinglish-Swift --task automatic-speech-recognition-with-past ./out
2. Encoder converted to fp16 via onnxconverter_common.float16.convert_float_to_float16 (keep_io_types=True).
3. Decoder kept as the unmodified fp32 decoder_model_merged.onnx produced by step 1.