Downloads · 30 days
130
87% of all-time downloads
astro-code-switch-0.5_7.9B
astroanand/astro-code-switch-0.5_7.9B
astro-code-switch-0.5_7.9B is a machine learning model from astroanand. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as gemma.
4-bit decoder-only quantization of Gemma 4 E4B, for on-device code-switched (Hindi–English) dictation. Built for Astro.
All-time downloads
149
Public
Parameters
7.9B
5.2 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors5.1 GB · 99%
Parameter types
How the weights are stored.
U327.5B · 94%
Base models
Model card
From the Hugging Face model README
astro_code_switch0.5_7.9B
4-bit decoder-only quantization of Gemma 4 E4B, for on-device code-switched (Hindi–English) dictation. Built for Astro.
The audio encoder is left at bf16 on purpose. Only the language decoder is quantized — it is 94% of the weights, so it is the whole lever, and quantizing the encoder costs transcription quality for almost no size saving.
| component | params | share | precision |
|---|---|---|---|
language_model (decoder) | 7.46 B | 94.1% | 4-bit, group 64, affine |
audio_tower (encoder) | 0.305 B | 3.8% | bf16 |
vision_tower (unused for dictation) | 0.167 B | 2.1% | bf16 |
| total | 7.93 B |
Measured results
40 clips, same eval script and prompt, run one at a time on an M5 Pro (48 GB).
| variant | on disk | WER | CER | median latency | RTF |
|---|---|---|---|---|---|
| bf16 baseline | 15.9 GB | 3.59% | 1.41% | 749 ms | 7.3× |
| this model (4-bit decoder) | 4.82 GB | 3.76% | 2.05% | 386 ms | 14.8× |
| 3-bit decoder | 3.96 GB | 260% | 296% | 373 ms | 12.6× |
3.3× smaller and ~2× faster for +0.17pp WER.
The 3-bit row is the useful part: quantization does not degrade gently here, it falls off a cliff. At 3 bits the model enters runaway repetition loops and is unusable. 4-bit is the frontier, not a midpoint.
Why this exists
Parakeet is faster and more accurate on English (1.35% WER, 90 ms) but cannot speak Hindi or Gujarati. Gemma 4's audio encoder is genuinely cross-lingual and holds code-switched speech intact — English stays Latin, Hindi stays Devanagari, in the same utterance:
Paneer stock में available है, या नहीं confirm करो।
That property is what this model is for. The bf16 original is too heavy to keep resident (15.9 GB, 749 ms); this makes it practical on a laptop.
Usage
from mlx_vlm import load, generate
model, processor = load("astroanand/astro_code_switch0.5_7.9B")
Script policy (Devanagari ↔ Roman) is handled after transcription by a deterministic normalizer, not by prompting. Prompt steering was measured and does not work on this family: the 0.6B is steerable but unstable, the 1.7B ignores prompts entirely. Romanization is only safe in the Indic→Latin direction; Latin→Indic is destructive.
Provenance
Quantized from mlx-community/gemma-4-e4b-it-bf16 with mlx_vlm. Reproduction scripts
(01_inspect_split.py, 02_quantize.py, 03_eval.py) and full results are in the
Astro research tree. Original Gemma terms of use apply.
- Library
- mlx
- Type
- gemma4
- License
- gemma
- Languages
- en, hi
- Created
- Aug 29, 2026
- Updated
- Aug 29, 2026