Breeze-ASR-25-mlx-4bit

Breeze-ASR-25 · MLX (4-bit)

This repository contains MediaTek-Research/Breeze-ASR-25 converted to MLX, Apple's array framework for Apple Silicon, and quantized to 4 bits. The source model is a Whisper-large-v2 fine-tune for Taiwanese Mandarin and Mandarin–English code-switching.

The weight file is 0.88 GB against the source model's 3.09 GB, and the model transcribes a five-minute recording in 17 seconds using 1.8 GB of memory. The same recording takes 168 seconds and 28.2 GB through the source checkpoint on transformers, and 24 seconds and 4.0 GB through the float16 build of these same weights.

Quantization is not free, and its cost here depends on the material. On conversational speech this build scores 1.2 percentage points worse than the float16 build and drops roughly twice as many utterances entirely; on a 57-minute lecture recording the two are within 0.1 points of each other. §3 gives the measurements and §3.5 the comparison.

What the model can do, how it was trained, and its known biases are documented by its authors and are not repeated here.

---

1. Usage

With mlx-audio, and servers built on it

The repository is laid out the way mlx-audio expects, so no preparation is needed:

from mlx_audio.stt.utils import load_model

model = load_model("BRlin/Breeze-ASR-25-mlx-4bit")
print(model.generate("audio.wav", language="zh").text)

With mlx-whisper

mlx-whisper 0.4.3 — the current PyPI release — reads a weight file named weights.safetensors, while servers built on mlx-audio require the name model.safetensors when they scan the local HuggingFace cache. A single file cannot carry both names. This repository ships model.safetensors, following the convention OpenAI's own Whisper repositories use, so users of the released mlx-whisper create an alias once after downloading. A hard link is a second directory entry pointing at the same data, so it costs no additional disk space:

cd <downloaded-directory> && ln model.safetensors weights.safetensors

This step is specific to the released version. mlx-whisper on its main branch already prefers model.safetensors (PR #1399, merged 2025-12-15); verified against a build from that branch, this repository loads with no alias. The change is not in any published release — main still reports version 0.4.3 — so the step remains necessary until one ships.

import mlx_whisper

r = mlx_whisper.transcribe(
    "audio.wav", path_or_hf_repo="<downloaded-directory>", language="zh",
)
print(r["text"])

mlx-whisper reconstructs the quantized layers from the quantization block in config.json before loading weights, so no additional argument is needed.

Repeated runs will not produce identical text

mlx_whisper.transcribe() re-decodes a segment at successively higher temperature when the segment fails its compression-ratio or log-probability checks. Temperature above zero means the decoder samples from the token distribution rather than taking its most probable token, so two runs over the same audio can differ. Two runs of the full ASCEND test set differed by 0.07 percentage points here. Passing temperature=0.0 removes the fallback and makes output identical across runs; it also removes the mechanism that limits repetition loops on silence, so it suits benchmarking rather than unattended transcription.

Long or noisy recordings

Whisper models, including this one, emit repeated tokens when they encounter silence under the default thresholds. Tightening those thresholds reduces it:

r = mlx_whisper.transcribe(
    "audio.wav", path_or_hf_repo="<downloaded-directory>", language="zh",
    condition_on_previous_text=False,  # do not carry context across windows
    compression_ratio_threshold=1.8,   # tighter than the 2.4 default
    logprob_threshold=-0.6,            # enter temperature fallback sooner
    no_speech_threshold=0.5,
    hallucination_silence_threshold=2.0,
)

No measurement in §3 used these values; each table there states the decoding settings it was produced under.

What does not work

transformers cannot load this repository. The tensor names follow MLX conventions rather than a PyTorch state dictionary, and the weights are stored in MLX's quantized layout.

---

2. What the conversion did

The source checkpoint holds 1,259 tensors in bfloat16 under HuggingFace naming. This build holds 2,284 tensors under MLX naming: 1,771 in float16 and 513 as packed 32-bit integers. The count rises because a quantized layer is stored as packed integer weights alongside the scale and offset that reconstruct them, where the source stored one float tensor.

Quantization is affine at 4 bits with a group size of 64, meaning each consecutive group of 64 weights shares one scale and one offset. Averaged over the whole model, including the layers left at float16 and the per-group parameters, the file stores 4.576 bits per weight.

One tensor present in the source is absent here, as in the float16 build: model.encoder.embed_positions.weight. MLX generates the encoder's positional embedding when the module is constructed rather than reading it from the file.

Conversion used mlx-audio 0.4.2 on an Apple M4 Pro running macOS 26.2. Two departures from a stock invocation apply to anyone reproducing this build.

The config.json the converter writes is a verbatim copy of the source repository's HuggingFace configuration. mlx-whisper parses config.json into a ten-field structure and raises TypeError on any field outside it, so the file was rewritten into that schema — retaining the quantization block, which mlx-whisper reads and removes before parsing the rest. A quantization_config key, which some tooling writes alongside it, must not be kept: mlx-whisper does not remove that one, and it reaches the parser and raises.

The converter's --dtype argument does not reach the model constructor, so the unquantized tensors are float16 whatever value is supplied. That is the intended format here, but the argument cannot be relied on to select another.

---

3. Measurements

3.1 Evaluation data

Accuracy is measured on two public corpora. Both appear in the upstream card's results tables, so the figures below can be placed beside the ones its authors publish.

CAiRE/ASCEND is a corpus of spontaneous Mandarin–English conversation with expert transcripts, released under CC BY-SA 4.0. The complete test split is used: 1,315 utterances totalling 55.0 minutes, with a median utterance of 1.9 seconds and none longer than 11.8 seconds. Each utterance carries a language label — 685 zh, 257 en, 373 mixed — and results are broken out by that label. Its reference transcripts are written in Simplified Chinese.

ky552/ML2021_ASR_ST holds recordings from National Taiwan University's 2021 Machine Learning course: Mandarin lecturing carrying English technical vocabulary. The upstream card reports it under the name ML-lecture-2021-long. The released test split is 14,916 short clips tagged with the lecture they came from; concatenating one lecture's clips in released order reconstructs continuous audio. Lecture week7_1 is used here, 1,676 clips totalling 57.4 minutes, with its first 300 seconds serving as the throughput excerpt. Its reference transcripts are written in Traditional Chinese. The dataset card declares no licence; the upstream table lists the corpus as MIT.

3.2 How error rates were computed

The upstream card reports word error rate, the edit distance between transcript and reference divided by the number of reference words. Counting words in Mandarin–English text requires deciding where word boundaries fall, and the decision made upstream is not published. The figures here therefore use a mixed error rate: each Chinese character counts as one unit, each unbroken run of Latin letters or digits counts as one unit, and the edit distance over that sequence is divided by the number of reference units. Punctuation is removed, case is folded, and [UNK] markers — annotations for speech the transcriber could not resolve, not words the model should produce — are deleted from the reference.

This model writes Traditional Chinese. ASCEND's references are Simplified and NTUML2021's are Traditional, so hypotheses are converted to Simplified before scoring against ASCEND and left unconverted for NTUML2021.

3.3 Accuracy

The table places this build beside the figures the upstream authors publish for the unquantized model. Both columns are error rates, so lower is better; Δ is their difference in percentage points. The upstream column is word error rate as published; ours is the mixed error rate described above, and §4 discusses what that difference in definition permits. Rows use the default decoding parameters and are the mean of two runs, which differed by 0.07 points overall.

Part of that difference belongs to quantization and part to the change of runtime. The float16 build of these same weights, measured identically, scores 18.01 % overall, so quantization accounts for 1.19 points of the difference and the remaining 0.27 sits between the MLX and PyTorch execution paths.

On long-form audio the same measurement over the reconstructed 57.4-minute lecture gives 5.82 % for this build, against 5.91 % for the float16 build and the 4.98 % upstream reports for ML-lecture-2021-long. Upstream does not state which lectures that figure covers and this measurement uses one of fourteen, so part of that gap is a difference in material rather than in the model.

3.4 Utterances the model returns empty

Error rate counts substituted, inserted and deleted units, which means a transcript the model declines to produce at all and a transcript with several words wrong can contribute similarly. They are not equivalent in use, so the two are counted separately here. The figures are from the full ASCEND test split under deterministic decoding; the harness records inference failures separately, and neither run had any, so these are cases where the model ran and returned nothing.

This build returns nothing about twice as often. ASCEND contains many very short utterances — its median is 1.9 seconds, and single filler words such as 呃 and um appear as complete items — which is the material most affected. If your audio consists of short isolated utterances rather than continuous speech, this behaviour matters more than the error rate difference does.

3.5 Throughput and memory

Speed is reported as real-time factor: processing time divided by audio duration, so 0.06 means a minute of audio takes under four seconds and lower is faster. Peak memory is the highest allocation observed during the run.

Throughput on this machine depends on what ran before it — a configuration measured second on a warm device runs roughly 1.3× slower than the same configuration measured first — so the configurations were run in order and then immediately in reverse, giving each the same average position. Each figure is the median of four timed runs on a 300-second excerpt of the public lecture recording, with a warm-up run discarded.

Against the source checkpoint this build processes the excerpt 10.0× faster in 15.7× less memory; against the float16 build, 1.4× faster in 2.2× less memory.

The source row runs in float32 because transformers with float16 on MPS — PyTorch's Metal backend — produces unusable output for this architecture. float32 is the only working configuration on that path, and its memory figure reflects that rather than a property of the model.

These figures describe a five-minute recording. Over the full 57.4-minute reconstruction this build measured RTF 0.072 at 3.43 GB peak, so both cost and memory were higher on the longer material.

3.6 Choosing between this and the float16 build

Accuracy rows use temperature=0.0 on both builds, which makes each figure exact rather than an average over runs that differ.

The two separate by 1.29 points on ASCEND's short conversational utterances and by 0.09 points on the lecture recording, in opposite directions. A single lecture is not enough to establish that they are equivalent on long-form material, but the cost of quantization is clearly not constant across the two kinds of audio. The empty-transcript difference in §3.4 persists in both settings.

---

4. Limitations

Coverage. Two corpora, both Mandarin–English. Nothing here measures other languages, other speech registers, or the Formosa long-form corpora the upstream card also reports. The long-form figure rests on one lecture of fourteen.

Metric comparability. The word segmentation behind the published upstream figures is not available, so §3.3 places two similarly-defined but not identical metrics side by side. That matters more for this build than for the float16 one, because the difference being measured is larger relative to the uncertainty.

Attribution of the difference. §3.3 separates quantization from runtime by comparing against the float16 build, but both were measured on the same MLX path, so that separation assumes the runtime contributes equally to both.

Runtime coverage. Accuracy was measured through mlx-whisper only.

Version coverage. Behaviour was verified against mlx-audio 0.4.2 and 0.4.5. Version 0.4.6 and later were not tested.

Not measured. Word-level timestamps, including the alignment quality the upstream card describes; streaming or real-time operation; accuracy under the tightened thresholds shown in §1; quantization at bit widths other than 4.

Limitations that belong to the base model rather than to this conversion are documented upstream.

---

5. Licence and attribution

Apache-2.0, inherited from the source model. This build adds no restrictions.

The model is the work of MediaTek Research. This repository contributes the MLX build and the measurements above.

• Model: MediaTek-Research/Breeze-ASR-25 · paper · code
• Base architecture: openai/whisper-large-v2
• float16 build of these weights: BRlin/Breeze-ASR-25-mlx-fp16
• Evaluation data: CAiRE/ASCEND · ky552/ML2021_ASR_ST
• Runtimes: ml-explore/mlx-examples · Blaizzy/mlx-audio

@article{breeze-asr-25,
  title   = {A Self-Refining Framework for Enhancing ASR Using TTS-Synthesized Data},
  journal = {arXiv preprint arXiv:2506.11130},
  year    = {2025}
}