Ling-3.0-tiny-MLX-4bit

Ling-3.0-tiny — MLX 4-bit

The first MLX conversion of inclusionAI/Ling-3.0-tiny: a 7.9B-total / 1.3B-active sparse-MoE reasoner (128 experts, top-8 + 1 shared) with a KDA + MLA hybrid attention stack and 131K context, MIT licensed.

4.2 GB at 4.507 bits/weight — it fits and runs on an 8 GB Apple Silicon Mac.

Serve it

The bailing_hybrid architecture is not in upstream mlx-lm yet — this checkpoint is served by rapid-mlx, which ships a verified native implementation (reference parity 1.5e-6 against the official modeling code):

pip install -U rapid-mlx   # 0.12.10 or newer
rapid-mlx serve ling-3.0-tiny-4bit

You get an OpenAI-compatible server on localhost:8000 with reasoning (reasoning_content) and tool calling parsed natively — thinking is controlled with chat_template_kwargs: {"enable_thinking": true} or the model's detailed thinking on/off system-prompt switch.

Once mlx-lm gains native bailing_hybrid support, this checkpoint will load there unchanged.

Conversion provenance

Converted with mlx_lm.convert (quantize=True, q_bits=4, q_group_size=64) running rapid-mlx's vendored bailing_hybrid implementation (PR #1817), which was verified against the official modeling_bailing_moe_v3.py on identical random weights to a max logits deviation of 1.5e-6 (full prefill) / 1.9e-6 (token-by-token incremental) before conversion. End-to-end chat / reasoning / tool-call behaviour validated on an M2 Pro Mac mini.