Qwen3.8-27B-MTP-4bit

EigenLabs/Qwen3.8-27B-MTP-4bit

This is a non-standalone, one-layer native MTP proposal head for the pinned target backbone EigenLabs/Qwen3.8-27B-4bit@301e9e2767fd0efcfab7883004720ba3c9a552a1. It proposes tokens only. The target backbone remains authoritative for language tensors, embeddings, the LM head, token acceptance, and final output.

The official model lineage is Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. This artifact was derived locally from the exact EigenLabs BF16 MTP artifact EigenLabs/Qwen3.8-27B-MTP-bf16@26a328e070875b0314d652a039b6b59902690f03, never by copying a third-party quantized checkpoint.

Artifact contract

• model_type: qwen3_5_mtp
• MTP layers: 1 (layers.0 only)
• block_size: 3
• 31 tensor keys, all bare (no mtp. prefix)
• Eight 2-D matrices stored as canonical .weight, .scales, and .biases tensors
• Seven 1-D normalization tensors retained in BF16
• No tokenizer, target embeddings, target LM head, target-backbone weights, or vision weights

This repository must be paired with the immutable target revision above. Updating or downloading this repository alone does not register it in a runtime catalog or enable speculative decoding.

Quantization

Quantization used MLX 0.31.2 directly:

weight, scales, biases = mlx.core.quantize(
    source_bfloat16, group_size=64, bits=4, mode="affine"
)

The packed weights are uint32; affine scales and biases remain BF16. MLX-LM 0.31.3, safetensors 0.7.0, and huggingface-hub 1.7.2 were installed in the conversion environment. Each matrix was independently dequantized with mlx.core.dequantize(..., group_size=64, bits=4, mode="affine", dtype=mlx.float32) and compared with its exact BF16 source.

All source, packed, scale, bias, and dequantized values passed finite-value checks. After the local derivation was complete, all 31 local tensor payloads were compared with the independent pinned reference mlx-community/Qwen3.8-27B-MTP-4bit@b643c01b6d3b094e325edb6ebd832e16c486c575; names, dtypes, shapes, and raw tensor bytes matched exactly.

These checks establish artifact structure and quantization fidelity only. They are not end-to-end generation, acceptance-rate, quality, or speedup claims.