Kokoro-82M-CoreML

Kokoro-82M for Core ML

Kokoro-82M converted to Core ML as two packages, with variable-length inputs — no padding to fixed buckets, and no cap on utterance length below the model's own 512-phoneme context.

It also returns a duration for every phoneme, so word- and phoneme-accurate timing comes out of the same pass as the audio. That is the timing AVSpeechSynthesizer declares and does not report.

macOS 15+ / iOS 18+. FP16. 24 kHz output.

Files

Voices.bin holds the 54 published voices as 510 style vectors each — one per phoneme count — flat and memory-mappable:

magic     8   "AIOSVOX" and a version byte
count     4   voices                     lengths   4   vectors per voice (510)
dimension 4   numbers per vector (256)   reserved  4
names  24·n   ASCII, NUL-padded
data      …   count · lengths · dimension float32, little-endian

Why two packages

Between predicting durations and rendering audio, Kokoro repeats each phoneme's features by the number of frames it predicted for that phoneme. Every shape downstream then depends on a value the model computed. Core ML has no such shape; ONNX hides it behind a dynamic axis.

The split falls exactly there, and the step that moves to the caller is an index gather — arithmetic stays in the graph.

  tokens, style ──▶ KokoroProsody ──▶ prosody, text, duration
                                            │
                    gather (caller) ◀───────┘   repeat column i, duration[i] times
                            │
  prosody', text', style ──▶ KokoroAcoustic ──▶ audio

Interfaces

KokoroProsody — tokens [1, 3…512] int32, style [1, 256] fp32 → prosody [1, 640, T], text [1, 512, T], duration [1, T] fp32.

Tokens are the Kokoro phoneme vocabulary with token 0 at both ends. style is the row of Voices.bin for the phoneme count, excluding those two.

duration is in frames, unrounded and unscaled — round to at least 1, and divide by speed here rather than re-running the model.

KokoroAcoustic — prosody [1, 640, F], text [1, 512, F], style [1, 256] → audio [1, F·600] fp32 at 24 kHz.

F is the sum of the rounded durations. One frame is 600 samples, so 40 frames per second.

The phonemizer

G2PEncoder and G2PDecoder are a converted PeterReid/graphemes_to_phonemes_en_us — a one-layer BART trained on the misaki dictionaries, so it emits the same alphabet the voice reads. 2.8 MB together. It is for the words a dictionary does not have: about 1.5% of ordinary text, mostly proper nouns and technical terms.

Encoder — input_ids [1, 32] int32, attention_mask [1, 32] int32 → encoder_hidden_states [1, 32, 128].

Decoder — decoder_input_ids [1, 32] int32, encoder_hidden_states, encoder_attention_mask [1, 32] int32 → logits [1, 32, 63].

Both are fixed at 32 and padded. Feed the encoder bos + graphemes + eos; start the decoder with decoder_start_token_id at position 0, read logits[step], write the argmax to position step + 1, stop at eos. There is no key-value cache — re-running a one-layer 128-wide decoder is cheaper than keeping one.

g2p-vocab.json carries graphemeChars for the input. Do not decode the output through it. Input and output share one embedding and agree only up to id 25; above that one alphabet is spelling and the other is phonemes. Read the output through the phoneme table or "loosen" comes back as lTusXn — real phonemes wearing the wrong letters, which looks exactly like a broken model and is not one.

Scored against the dictionary it was trained on: 63% exact, 9% phoneme error, matching the PyTorch original it was converted from.

Fidelity

Spectral correlation 0.977 against the PyTorch reference on a held-out utterance — magnitude spectra, not waveforms. Waveform correlation is not a usable measure for this model: the vocoder adds random noise to its unvoiced excitation, so two runs of the reference do not match sample for sample either.

For calibration, on the same utterance: two reference runs differing only in that noise score 0.998, and a conversion with the modulo bug described in NOTICE scores 0.801 while producing audio of the right length, the right envelope and the right word timing. If you convert this model yourself, check spectra.

Speed

Apple M4 Pro, 3 s of audio, median of warm calls, each stage on its best unit:

47 ms in total, about 63× realtime. Set compute units per model — the default placement runs the prosody half 5.6× slower than the CPU alone does, because it is a small BERT-and-LSTM graph over a few dozen tokens and shipping that to another unit costs more than the work.

Neither package runs with the GPU excluded; variable shapes keep them off the Neural Engine. On a machine with a weaker GPU, a bucketed conversion such as mattmireles/kokoro-coreml may do better. Measure on your target.

Loading and compiling both packages costs about 0.35 s once.

Licence and attribution

Apache-2.0, inherited from Kokoro-82M. NOTICE lists the seven conversion changes, as section 4(b) requires — no weights were retrained, fine-tuned or altered. Voice vectors are redistributed from the kokoro-onnx model-files-v1.0 release.