G9v3-39A5B-MLX-4bit

Run it on your Mac

curl -fsSL https://rapidmlx.com/install.sh | bash
rapid-mlx serve g9v3-39a5b-4bit
curl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"default","messages":[{"role":"user","content":"Say hello"}]}'

Use it from OpenCode / Claude Code / Codex: rapid-mlx agents <name> --setup · Agent setup

Download the Desktop app · 32 GB unified memory catalog minimum; the weights alone are 22.8 GB.

G9v3-39A5B — MLX 4-bit

MLX conversion of ai9stars/G9v3-39A5B, a 39B Mixture-of-Experts causal LM with 5B parameters active per token (32 of 320 routed experts + 1 shared expert), gated GQA attention and a 128K context.

The g9v3 architecture is not in mlx-lm; it is served by the vendored MLX backbone in rapid-mlx (vllm_mlx/models/g9v3.py), which registers itself into mlx-lm's model lookup. Plain mlx_lm.load without rapid-mlx will not recognise the model_type.

Quantization recipe (mlx_lm.convert(..., quantize=True, q_bits=4, q_group_size=64), mlx-lm 0.31.3, with the vendored module registered): the 320 routed experts per layer are 4-bit / group 64 (affine); attention, the dense first-layer MLP, the shared experts, embeddings and lm_head are 8-bit / group 64 — 4.681 bits per weight overall. Uniform 4-bit was measured to hurt this architecture's attention projections badly (mean top-1 agreement with the bf16 reference over all prompt positions 0.80 → 0.90 with this recipe; 8-bit everywhere 0.96, bf16 MLX itself 0.955). The MLX port was checked against the upstream trust_remote_code implementation on identical random weights (max |Δlogit| < 1e-5 in fp32) and on the released weights (see the rapid-mlx PR for the bf16 agreement tables).

Usage

curl -fsSL https://rapidmlx.com/install.sh | bash
rapid-mlx serve g9v3-39a5b-4bit

Then talk to it through the OpenAI-compatible API on http://localhost:8000. Reasoning uses Qwen3-style <think> blocks (enable_thinking toggles it); tool calls use the model's native XML format, parsed server-side.

License

Apache-2.0, following the upstream release.