mini-coder-4b-OptiQ-4bit

mlx-community/mini-coder-4b-OptiQ-4bit

A 4-bit mixed-precision MLX quant produced by mlx-optiq — the sensitivity-aware quantization toolkit for Apple Silicon. It beats stock uniform 4-bit on the six-metric Capability Score (45.83 vs 42.74, +3.09) and is ~2.2x more faithful to the original bf16 weights (KL 0.06 vs 0.13).

A 4-bit mixed-precision MLX quant of ricdomolm/mini-coder-4b (a code/agent model fine-tuned from Qwen3-4B-Instruct-2507). Per-layer bit-widths come from a KL-divergence sensitivity pass on a six-domain calibration mix (prose · reasoning · code · agent · tool-call · constraint-bearing instructions). Sensitive layers go to 8-bit; robust ones stay at 4-bit.

Quantization details

We follow the same naming convention llama.cpp uses for Q4_K_M and similar mixed-precision quants: the "4-bit" label is for the predominant precision, not the weighted average. The few sensitive layers held at 8-bit make this build +0.7 GB larger than a stock uniform-4-bit quant (2.8 GB vs 2.1 GB) while recovering quality the uniform quant loses.

Usage

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/mini-coder-4b-OptiQ-4bit")
response = generate(
    model, tokenizer,
    prompt="Write a Python function to check if a string is a palindrome.",
    max_tokens=512,
)

It's a code/agent model — best paired with a lightweight agent harness / loop. For more (mixed-precision KV-cache serving, sensitivity-aware LoRA fine-tuning, OpenAI + Anthropic-compatible inference server with structured/JSON output and server-side tool-call healing, hot-swap mounted adapters), install mlx-optiq:

pip install mlx-optiq

Benchmarks

Six-metric Capability Score (mean of MMLU + GSM8K + IFEval + BFCL + HumanEval + HashHop). Apples-to-apples comparison against stock uniform 4-bit:

OptiQ's clearest wins are GSM8K (+11.5) and tool-calling (BFCL +9.0), plus markedly better fidelity to the original weights. IFEval and HumanEval are within noise (~±7 pp at these sample sizes). HashHop is 0% for both quants: a genuine model-capability gap — the model has no long-context multi-hop retrieval ability (verified by inspecting outputs, it reasons in prose instead of walking the hash chain) — not a quantization artifact, which is why OptiQ and uniform-4 match there.

Every metric gets one equal vote; disk size is an honest second axis, not folded into the score. See the eval-framework writeup.

Links

• Project website: mlx-optiq.com
• Docs: mlx-optiq.com/docs
• PyPI: pypi.org/project/mlx-optiq
• Eval framework: mlx-optiq.com/blog/eval-framework
• Base model: ricdomolm/mini-coder-4b

License

MIT (inherits from the base model).