Gemma-4-E4B-it-Text-MLX-Mixed-3.40bpw

Gemma 4 E4B-it Text MLX Mixed 3.40bpw

TEXT-ONLY MLX QUANTIZATION

An experimental text-only MLX affine mixed-precision quantization of google/gemma-4-E4B-it. This is my first public quantization at this model scale, so practical feedback from people who try it is especially welcome. It is intended for local text generation on Apple Silicon using MLX-LM; feedback and discussion are welcome through the maintainer account maglun.

This is a compressed model, not a claim of lossless behavior or a new quality frontier. It was selected as the project's compact validated E4B point and evaluated against the bf16 E4B H0 comparator with thinking enabled.

Fig 1. Bit allocation of model as shown on TensorLens (see link for details)

Quickstart

pip install mlx-lm
python -m mlx_lm.generate \
  --model maglun/Gemma-4-E4B-it-Text-MLX-Mixed-3.40bpw \
  --prompt "Explain why the sky appears blue." \
  --temp 0

The packaged current official Gemma 4 chat template is used by default, with thinking enabled by the tested chat path. For programmatic use, use the standard mlx_lm APIs and the tokenizer files included here. The tested runtime was MLX 0.32.0, MLX-LM 0.31.3, Python 3.12 on an Apple M4 Pro with 64 GB memory.

The current official template includes thinking-channel formatting and tool-call/tool-response message formatting. Tool calling was not a separately scored capability benchmark in this release, so the tool-calling tags describe the packaged interface/template support rather than a quality or reliability guarantee for autonomous tool use.

Measured resources

These are measurements on one Apple M4 Pro Mac mini, with thinking enabled, greedy temperature 0, batch size 1, fresh processes, and no intentional concurrent benchmark load. They are not universal speed claims.

Prefill and decode rates come from stock mlx_lm.stream_generate telemetry. Decode rate is not derived from total wall time. Full machine context is retained in the private release evidence.

Evaluation

Release-time evaluation used the current official E4B template, greedy temperature 0, thinking enabled, the same prompt/scorer harness for candidate and bf16 H0, and no optional comparator. All scores use frozen full denominators; operational failures remain incorrect.

Operational anomalies were retained in the denominator: one candidate IFEval cap hit, five H0 MMLU-Pro watchdog timeouts, and one candidate HumanEval+ evaluator timeout. There were no GSM8K anomalies.

†Totals sum the displayed rows, while the difference is the unweighted average of the seven displayed percentage-point differences. The four IFEval views are counted separately; this descriptive aggregate is not a QRS or a single-suite score.

Artifact identity and lineage

• Upstream weights: google/gemma-4-E4B-it@a4c2d58be94dda072b918d9db64ee85c8ed34e3f.
• Active release metadata/template: current E4B revision ee0ef6023621cff504d758262d4e04895a5af4a2.
• Active chat_template.jinja SHA-256: 0a2c8073c878ab1da004bee933a998606537bbb62016310352c7285c3f01c5b5.
• Historical artifact template SHA-256: 2f1b4d75d067bae3fe44e676721c7f077d243bc007156cb9c2f8b5836613d082; it is retained only as clearly inert reference material and is not the runtime template.
• Format: MLX affine mixed-precision safetensors. GPTQ describes a rounding/build stage in the lineage, not AutoGPTQ compatibility.
• Effective size: 3,172,678,276 safetensors bytes; effective 3.4007947088485322 bits/weight, labelled 3.40 bpw using a denominator of 7,463,013,418 parameters.
• Weight SHA-256: 8de1a304992cc0823b5adfa9acf8ca0be00ab55561a48b925d8f28bc3b990759.
• Canonical safetensors-set SHA-256: 6624ef34a24eaa2ac2ebac7a1d9905280b1a5ff6ee07957781a0eb72ae92c044.

The quantization recipe, source provenance, file hashes, and package manifest are included with this release. This model is explicitly text-only: no image, audio, or multimodal processor support is included.

How this quantization was made

The public lineage is: bf16 text extraction H0 → AWQ fold → KGB mixed-precision allocation → GPTQ-style rounding/build stage → MLX affine conversion. The policy and calibration hashes, parameter denominator, and output identity are recorded in quantization_recipe.json and the release evidence.

Limitations and license

Quantization changes model behavior, including long-form completion and benchmark outcomes. The compact point shows measurable reductions versus H0 on MMLU-Pro and HumanEval+, while GSM8K remains close in this evaluation. Hardware testing covers one Apple M4 Pro configuration. This is an experimental research preview with no company or product support and no additional safety tuning beyond the upstream model. Do not infer broad safety, multilingual, or production-readiness claims from these results.

The package is released under Apache-2.0 terms applicable to the upstream model; see LICENSE. Gemma usage remains subject to Google's Gemma 4 license terms, including applicable attribution and usage requirements.

Citation and feedback

Please cite the upstream Gemma 4 E4B-it work and identify this repository and exact revision when reporting results. Feedback and corrections are welcome through the maglun maintainer account and repository discussions. This release is related to, but not directly score-comparable with, the earlier Serial 0 E2B release because the model sizes and weights differ.