Ornith-1.5-9B-mtp-drafter-mlx-4bit

MTP drafter for ornith-ai/Ornith-1.5-9B

The native multi-token-prediction head of ornith-ai/Ornith-1.5-9B, extracted as a standalone MLX drafter and quantized to 4-bit affine (group 64). It carries no embedding or LM head of its own — it binds to the target model's at load, so it pairs with any quantization of the same base model.

Speculative decoding verifies every proposed token against the target model, so this drafter changes throughput only, never output: greedy decoding is byte-identical with and without it.

Measured

Against ornith-ai/Ornith-1.5-9B served by MLX (mlx-vlm 0.6.17, 4k context, greedy, block_size 3):

Greedy output is byte-identical with and without the drafter at both quantizations. The -4bit in the name describes this artifact's own weights, not which targets it serves — one drafter covers every quantization of the base model, and a matched-precision drafter measured no better (an 8-bit build of this same head scored 2.34 / 1.96 on the same two targets, i.e. within noise for 1.8x the download).

Reproducing

Built by scripts/build-mtp-drafter.mjs from Gezel — a local-first desktop app for assembling a team of AI agents that run on your own machine:

git clone https://github.com/bendyline/gezel.git
node scripts/build-mtp-drafter.mjs --source ornith-ai/Ornith-1.5-9B --out <dir>

The script fetches only the checkpoint shard(s) carrying the mtp.* tensors (1 of 4 for this model), splits the head into a standalone drafter, and quantizes it to 4-bit.

License

mit, inherited from ornith-ai/Ornith-1.5-9B. Weights are a derivative of that checkpoint.