Downloads · 30 days
19
14% of all-time downloads
granite-speech-4.1-2b-mlx-4bit
divydeep/granite-speech-4.1-2b-mlx-4bit
granite-speech-4.1-2b-mlx-4bit is a automatic speech recognition model from divydeep. Use it when you need speech turned into text. It is set up for mlx. The card lists the license as apache-2.0.
Memory-oriented MLX conversion of ibm-granite/granite-speech-4.1-2b for Apple Silicon. This is the autoregressive Granite Speech model, not the NAR variant.
All-time downloads
132
Public
Parameters
2.3B
2.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.6 GB · 100%
Parameter types
How the weights are stored.
U321.4B · 62%
Base models
Model card
From the Hugging Face model README
Granite Speech 4.1 2B MLX 4-bit
Memory-oriented MLX conversion of
ibm-granite/granite-speech-4.1-2b
for Apple Silicon. This is the autoregressive Granite Speech model, not the NAR
variant.
Quantization
This conversion uses post-training weight quantization without training, calibration data, or an importance matrix.
| Component | Precision |
|---|---|
| 16-layer Conformer speech encoder | BF16 |
| 2-layer Q-Former speech projector | BF16 |
| Eligible internal language-model linear layers | MLX affine 4-bit, group size 64 |
| Token embedding and language-model output head | BF16 |
| Norms, biases, and unsupported tensors | BF16 |
- Source revision:
de575db64086f84fdc79da4932d1076e965bc546 - Effective average reported by MLX: 8.911 bits per weight
model.safetensors: approximately 2.46 GB
Usage
pip install -U mlx-audio
python -m mlx_audio.stt.generate \
--model /path/to/granite-speech-4.1-2b-mlx-4bit \
--audio audio.wav \
--output-path transcript \
--format txt \
--prompt "transcribe the speech with proper punctuation and capitalization."
Validation
The checkpoint was strictly loaded by mlx-audio and run with greedy decoding
on IBM's bundled multilingual_sample.wav. It preserved every reference word
and accent, but omitted three French punctuation or hyphen marks compared with
the BF16 reference. This is a smoke test, not a complete WER benchmark.
Quantization can affect names, rare words, punctuation, casing, translation, keyword biasing, and difficult or noisy audio. Use the 8-bit variant when closer agreement with the original model is more important than memory usage.
License
Apache-2.0, matching the original model.
- Task
- Automatic Speech Recognition
- Library
- mlx
- Type
- granite_speech
- License
- apache-2.0
- Languages
- multilingual, en, fr, de, es, pt
- Created
- Jul 11, 2026
- Updated
- Jul 11, 2026