whisper-medical-large-onnx

Medical Whisper large-v3 — ONNX (q4, transformers.js)

4-bit ONNX export of Na0s/Medical-Whisper-Large-v3, a fine-tune of openai/whisper-large-v3 on doctor/patient consultations (Na0s/Primock_med). 1.55 B parameters, 32 encoder + 32 decoder layers, d_model 1280, 128 mel bins, vocab 51866.

Usage

import { pipeline } from '@huggingface/transformers';

const transcriber = await pipeline(
  'automatic-speech-recognition',
  'jbilcke/whisper-medical-large-onnx',
  { device: 'webgpu', dtype: 'q4' },
);

const { text } = await transcriber(audio, { chunk_length_s: 30, stride_length_s: 5 });

Base whisper-large-v3 is multilingual, so pass language explicitly to skip auto-detection.

Why q4 only

q8 (*_quantized.onnx) fails session creation on the onnxruntime-web bundled into transformers.js 4.x, and onnxruntime-web cannot be pinned separately because it ships inside transformers.web.js:

qdq_actions.cc:137 TransposeDQWeightsForMatMulNBits
Missing required scale: model.decoder.embed_tokens.weight_merged_0_scale

fp32 needs .onnx_data external-data sidecars — a 1.55 B-parameter fp32 graph exceeds protobuf's 2 GB serialization limit — which ORT Web does not handle cleanly. So q4 is both the smallest and the only browser-viable option.

Verification

Encoder and decoder both build an ONNX Runtime session, and q4 transcribes the reference clip identically to the fp32 export and to the PyTorch original.

Measured on CPU (Python onnxruntime 1.29, Apple M4, 11 s of audio, median of 3): q4 runs 17.0 s vs 7.1 s for fp32. That inversion is ORT's MatMulNBits CPU kernel, which dequantizes per 32-element block. On WebGPU it has a dedicated shader and 4-bit weights cut VRAM traffic, so the ordering is expected to reverse — unverified. Expect q4 on the wasm backend to be slower than real-time.

This is a single-clip smoke test, not a WER benchmark. It does not establish medical-transcription accuracy at 4-bit precision.

Quantization

MatMulNBitsQuantizer with DefaultWeightOnlyQuantConfig: 4-bit, block size 32, symmetric, QOperator format (com.microsoft.MatMulNBits, which ORT Web supports). The RTN and GPTQ configs cannot be used here — they route through neural_compressor, whose topological_sort assumes every graph input has a top-level consumer, and the merged decoder's only top-level node is the If whose branches consume those inputs inside subgraph attributes.

License

Apache-2.0, inherited from Na0s/Medical-Whisper-Large-v3 and openai/whisper-large-v3.