canary-1b-v2-pal6-int8-coreml

smdesai/canary-1b-v2-pal6-int8-coreml

canary-1b-v2-pal6-int8-coreml is a automatic speech recognition model from smdesai. Use it when you need speech turned into text. It is set up for coreml. The card lists the license as cc-by-4.0.

Smallest Apple Neural Engine-resident CoreML build of NVIDIA canary-1b-v2: a 6-bit k-means palettized encoder paired with an INT8 per-channel stateful KV-cache decoder and cross-KV. Accuracy matches the FP16 reference…

Downloads · 30 days

Access

Public

Updated Oct 12, 2026

Repo size

1.4 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin1.4 GB · 100%

At a glance

Task: Automatic Speech Recognition
Library: coreml
License: cc-by-4.0
Access: Public
Created: Sep 8, 2026
Updated: Oct 12, 2026
SHA: d5e16e66

Base models

nvidia/canary-1b-v2

Model card

From the Hugging Face model README

Canary 1B v2 — CoreML (6-bit encoder + INT8 decoder, KV-cache)

Smallest Apple Neural Engine-resident CoreML build of NVIDIA canary-1b-v2: a 6-bit k-means palettized encoder paired with an INT8 per-channel stateful KV-cache decoder and cross-KV. Accuracy matches the FP16 reference build (canary-1b-v2-coreml) at under half its RAM. Pick this over the all-INT8 build (canary-1b-v2-int8full-coreml) when the extra ~7 ms/window encoder cost and the longer first-load compile are acceptable.

Base model: nvidia/canary-1b-v2 (NVIDIA NeMo EncDecMultiTaskModel, 1B parameters, 25 European languages, ASR + speech translation). License: the base model is released under CC-BY-4.0; this conversion carries the same license. Please attribute NVIDIA for the model.

This build


Encoder	6-bit k-means palette, 16-channel groups; Neural Engine-resident, 614 MB on disk, ~0.63 GB resident
Decoder / cross-KV	INT8 per-channel, 136 MB + 17 MB; ~134 MB Neural Engine-resident
Download	0.77 GB
iOS RAM while transcribing	910 MB (measured) vs 1.9 GB for the FP16 build
Encoder latency	~43 ms per 15 s window (M3 Max, warm); decoder step 2.4–3.3 ms per token

Placement note: load with MLComputeUnits.cpuAndNeuralEngine; under .all the planner prefers the GPU for the INT8 decoder.

Build family

All 1B v2 builds share the same preprocessor, tokenizer, package layout, and decode contract; only the weight format of the encoder and/or decoder differs. Download sizes are as hosted on the Hub; iOS RAM was measured in an app while transcribing a 6-minute file.

Repo	Build	Encoder weights	Decoder weights	Download	iOS RAM
`canary-1b-v2-coreml`	FP16 (reference)	FP16	FP16	1.89 GB	1.9 GB (measured)
`canary-1b-v2-int8-coreml`	INT8 encoder	INT8 per-channel	FP16	1.10 GB	1.2 GB (measured)
`canary-1b-v2-pal6-coreml`	6-bit encoder	6-bit palette, g=16	FP16	0.92 GB	1.0 GB (measured)
`canary-1b-v2-int8full-coreml`	INT8 full	INT8 per-channel	INT8 per-channel	0.95 GB	1.0 GB (measured)
`canary-1b-v2-pal6-int8-coreml` (this repo)	6-bit + INT8	6-bit palette, g=16	INT8 per-channel	0.77 GB	910 MB (measured)
`canary-180m-flash-coreml`	180M Flash	FP16 (17L, d=512)	FP16 (4L)	0.37 GB	470 MB (measured)

Files

File	Contents
`canary_preprocessor.mlmodelc`	FP32 mel front end (128 mel bins, 16 kHz mono, 15 s window)
`canary_encoder.mlmodelc`	FastConformer encoder, 32 layers, d=1024, subsampling 8
`canary_cross_kv.mlmodelc`	Cross-attention K/V projection, 8 layers (run once per window)
`canary_decoder_kv.mlmodelc`	Stateful single-step Transformer decoder, 8 layers, 16384-way LM head
`canary_spe.model`	SentencePiece tokenizer (16384 pieces)
`metadata.json`	Shapes, decode geometry, seed tokens, and the weight recipe of this build

Word-level timestamps (optional)

The timestamp/ folder is only needed for word-level start and end times. Transcription and translation do not use it, so skip it if you only need text.

It contains NVIDIA's auxiliary CTC model from canary-1b-v2.nemo, a 24-layer Conformer, which NeMo uses for transcribe(..., timestamps=True). Run it on the same 15 s audio window, then force-align the Canary transcript against its scores with a Viterbi pass. Word boundaries have 80 ms resolution.

This repo ships the INT8 timestamp model (602 MB), which is not included in the download size above. INT8 keeps word timing at FP16 parity. Across 227 words in six English and French test windows, INT8 moves 5 words by one 80 ms frame relative to NVIDIA's PyTorch model; FP16 moves 3. No word moves by more than one frame. On a 15-second English clip, all 55 words match NeMo's own timestamps=True output exactly. On an iPhone 17 Pro (iOS 27.2), the INT8 encoder's compute plan matches FP16 (99.9% of estimated cost on the Neural Engine), and a warm 15 s window takes about 31 ms in the encoder, the same as FP16.

File	Contents
`timestamp/timestamp_preprocessor.mlmodelc`	Mel front end: `audio_signal` [1, 240000] float32 (16 kHz mono, zero-padded) and `audio_length` [1] int32 (unpadded samples) → `mel`, `mel_length`
`timestamp/timestamp_encoder.mlmodelc`	Conformer encoder, INT8 per-channel weights, 568 MB (the FP16 timestamp encoder is 1.1 GB) → `encoded` [1, 1024, 188], `encoded_length`
`timestamp/timestamp_ctc.mlmodelc`	CTC head → `log_probs` [1, 188, 16385]; column 16384 is blank; frames are 80 ms apart
`timestamp/tokenizer/`	SentencePiece tokenizer and vocabularies for the alignment targets (`tokenizer.model` is identical to `canary_spe.model`)
`timestamp/metadata.json`	Shapes, blank id, frame duration, and weight recipe
`timestamp/example-word-timestamps.json`	Reference word times for a 15 s English test clip

Load the timestamp models with MLComputeUnits.cpuAndNeuralEngine. Only frames up to encoded_length are valid.

Download without timestamps, or only the timestamp model:

hf download smdesai/canary-1b-v2-pal6-int8-coreml --exclude "timestamp/*"
hf download smdesai/canary-1b-v2-pal6-int8-coreml --include "timestamp/*"

Credits

Model: NVIDIA NeMo team, canary-1b-v2, CC-BY-4.0.

Task: Automatic Speech Recognition
Library: coreml
License: cc-by-4.0
Languages: bg, hr, cs, da, nl, en
Created: Sep 8, 2026
Updated: Oct 12, 2026