coreai-whisper-large-v3-turbo-kv-float16

Whisper large-v3-turbo — Core AI export (float16, KV cache)

An Apple Core AI export of OpenAI's whisper-large-v3-turbo, for on-device speech recognition on Apple silicon.

These are not PyTorch weights. The repository holds a single archive of .aimodel assets that run through Apple's Core AI runtime on macOS 27 / iOS 27 and later. They will not load with transformers.

Contents

whisper-large-v3-turbo-kv_float16.zip (~1.49 GB) unpacks to a bundle directory:

Configuration

• Architecture: Whisper encoder/decoder
• Precision: float16 · Cross-attention KV: packed
• d_model: 1280 · Decoder layers: 4 · Attention heads: 20
• Vocab: 51866 · Max target positions: 448
• Encoder window: 30 s (audio is padded to the window)
• Audio in: 16 kHz mono float32

Language

The weights are the full multilingual Whisper large-v3-turbo — this export is not English-only. What is English is one line of generation_config.json:

"forced_decoder_ids": [[0, 50258], [1, 50259], ...]   // 50259 = <|en|>

That token is the decoder prefix's language slot. It matters more than it looks: fed French audio, an <|en|> prefix does not fail, it transcribes the audio as if it were English and returns a fluent English translation. Measured on a French clip, that scores ~96% WER against a French reference while emitting a perfectly readable English sentence — a wrong language is silent unless you check for it.

Upstream apple/coreai-models has no API for changing the prefix, so with stock CoreAISpeech this bundle transcribes English only. The bjnortier fork adds language selection, and with it the same unmodified bundle handles every language Whisper knows:

// Detect the language from the audio (the default).
let (text, _) = try await model.transcribe(pcm: pcm, language: .detect)

// Or name it.
let (text, _) = try await model.transcribe(pcm: pcm, language: .code("fr"))

// Or keep the prefix this bundle shipped with.
let (text, _) = try await model.transcribe(pcm: pcm, language: .bundleDefault)

Detection costs one decoder step — Whisper's first prediction after <|startoftranscript|> is the language token — which is ~110 ms on a 62 s clip. For long-form audio the language is detected once, on the first window, and reused.

Verified on this bundle, unmodified, with --language auto:

No re-export is needed for other languages; the assets here already carry all 100 language tokens in added_tokens.json.

The prefix also pins <|notimestamps|>. Timestamps still require a different generation_config.json.

Usage

With CirceKit:

import CirceKit

let transcriber = CirceFileTranscriber(
    backend: .coreAI(.bundle(bundleURL)),
    locale: Locale(identifier: "en_US")
)
try await transcriber.prepare()
let result = try await transcriber.transcribe(fileAt: audioURL)
print(result.text)

Or directly with CoreAISpeech from apple/coreai-models:

let model = try await SpeechRecognitionModel(resourcesAt: bundleURL)
let (text, stats) = try await model.transcribe(audioURL: audioURL)

On the fork, CirceFileTranscriber passes its locale through as the decode language, so the example above transcribes French simply by asking for a French locale.

License and attribution

Released under the MIT licence, the licence of the source model. The original Whisper large-v3-turbo is by OpenAI; this repository only changes its serialization format. Refer to the upstream model card for training data, evaluation results, and intended use.