Downloads · 30 days
27
9% of all-time downloads
majentik/Gemma-4-E4B-MERaLiON-Speech-LoRA-MNSC-MLX
Gemma-4-E4B-MERaLiON-Speech-LoRA-MNSC-MLX is a automatic speech recognition model from majentik. Use it when you need speech turned into text. It is set up for mlx. The card lists the license as other.
A speech-language model for Singapore English ASR, built by composing:
Downloads · 30 days
27
9% of all-time downloads
All-time downloads
302
Public
Repo size
10.6 GB
Likes
0
Public
Click a slice to open those files.
.safetensors10.6 GB · 100%
From the Hugging Face model README
A speech-language model for Singapore English ASR, built by composing:
The whole bundle runs natively on Apple Silicon via mlx. The LoRA is switchable — disable it for plain text generation, enable it for speech transcription. No merging required.
Evaluated on a held-out Singapore English ASR test split (3000 clips):
| Model | WER ↓ |
|---|---|
| MERaLiON-2 (baseline, end-to-end speech-LM) | 25.78% |
| This model (Gemma-4-E4B + MERaLiON speech tower + ours) | 18.86% |
| Improvement | −6.92pp absolute (−26.8% relative) |
WER is computed with jiwer's standard normalization: lowercase, punctuation removed, whitespace collapsed. <SpeakerN>: dialogue prefixes (occasionally emitted by the decoder) are stripped before scoring — this happens natively inside transcribe_with_pipeline, not as a post-hoc evaluation hack.
This is a composite work. See PROVENANCE.md for the full chain-of-custody on every weight file.
| Component | Source | License |
|---|---|---|
| Decoder | majentik/gemma-4-E4B-RotorQuant-MLX-8bit ← google/gemma-4-E4B-it | Gemma Terms |
| Speech encoder + adaptor | majentik/MERaLiON-3-10B-MLX-4bit ← MERaLiON/MERaLiON-3-10B | MERaLiON-3 Public Licence |
| Projector | Trained by majentik for Singapore English ASR alignment | MIT |
| LoRA adapter | Trained by majentik for Singapore English ASR alignment | MIT |
| Composite work (this repo) | — | MERaLiON-3 Public Licence (most restrictive in chain) |
The composite is bound by the MERaLiON-3 Public Licence because we include MERaLiON-3 weights. Gemma's terms also apply to the decoder.
| Component | Storage dtype |
|---|---|
| Decoder weights | MLX 8-bit affine (uint32-packed) + bf16 norms/embeds |
| Speech encoder | fp16 |
| Adaptor | fp16 |
| Projector (ours) | fp32 |
| LoRA adapters (ours) | fp32 |
The decoder is the only quantized component. The speech tower stays at fp16 (it came that way from the MERaLiON-3-10B-MLX-4bit repo, where only the decoder shards are 4-bit). Our trained additions are at fp32 for maximum fidelity at minimal cost (208 MB combined).
.
├── config.json # composition manifest
├── PROVENANCE.md # full chain-of-custody
├── README.md # this file
├── decoder/ # Gemma-4-E4B 8bit MLX (~8.4 GB)
├── speech_encoder/ # MERaLiON-3 encoder + adaptor (~1.4 GB, fp16)
├── projector/ # our trained 3584→3072→2560 projector (75 MB, fp32)
└── lora/ # speech-alignment LoRA adapter (133 MB, fp32)
Total: ~9.9 GB.
Install the inference code:
pip install git+https://github.com/ajentik/elderwise-mlx.git
# or clone and pip install -e .
Then:
from elderwise.inference import load_pipeline, transcribe_with_pipeline
from huggingface_hub import snapshot_download
# Download the bundle
local = snapshot_download("majentik/Gemma-4-E4B-MERaLiON-Speech-LoRA-MNSC-MLX")
pipeline = load_pipeline(
meralion_dir=f"{local}/speech_encoder",
gemma_id=f"{local}/decoder", # local path, not HF id
projector_path=f"{local}/projector",
lora_path=f"{local}/lora",
lora_rank=16,
lora_target_names=(
"q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj",
),
)
text = transcribe_with_pipeline(pipeline, "your_audio.wav")
print(text)
The pipeline:
"Transcribe the following audio: " + projected speech embeddings + the PLE bypass (see Alignment notes).<SpeakerN>: prefix, returns the result.LayerNorm(3584) → Linear(3584→3072) → SiLU → Linear(3072→2560) → RMSNorm(2560). The final RMSNorm matters — Gemma-4 normalizes its embeddings to ‖x‖ ≈ 1.0 and projector outputs need to live on that manifold.The alignment adapter was trained on Singapore English ASR data. The public model card intentionally keeps internal optimization details high-level.
Gemma-4 uses Per-Layer Embeddings (PLE) as a side-channel into every transformer layer. When you call the model with inputs=None and only input_embeddings (as you must for speech, where there's no clean token id), mlx_lm runs nearest-neighbor recovery — it argmins your embeddings against the vocab table and uses the resulting fake token ids to populate PLE.
If your speech embeddings are off-manifold (they will be without the final RMSNorm), the nearest-neighbor returns garbage rare tokens, and Gemma reverts to its strongest priors (PII redaction templates, in our case — we measured 592% WER before fixing this).
The fix lives in elderwise/per_layer.py: we bypass nearest-neighbor by building per_layer_inputs explicitly — zeros for speech positions (silences the PLE side-channel where it has no signal anyway), and real embed_tokens_per_layer(token_ids) * scale for prompt/transcript positions.
This + three other compounding fixes (projector RMSNorm, prompt tokenization match, LoRA scale restore) is what made the model actually condition on speech. See the elderwise repo for the full diagnostic trail.
<SpeakerN>: artifacts are stripped, not used.)If you use this model, please cite:
@misc{gemma4_meralion_speech_lora_mnsc_mlx,
title = {Gemma-4-E4B-MERaLiON Speech LoRA for Singapore English},
author = {majentik},
year = {2026},
url = {https://huggingface.co/majentik/Gemma-4-E4B-MERaLiON-Speech-LoRA-MNSC-MLX}
}
And the upstream sources: