qwen3.5-0.8b-mtp-gguf

qwen3.5-0.8b-mtp-gguf (Enclave model volume)

Curated model volume for Enclave confidential inference: Qwen3.5-0.8B-MTP at Q8_0 (~795 MB), a single-file GGUF bundled with the official tokenizer.json so the volume is self-contained.

This is the SELF-DRAFTING variant of qwen3.5-0.8b-gguf: the GGUF carries the model's trained multi-token-prediction head (one extra layer), and eyesoff-ai's "draft": "mtp" setting uses it for speculative decoding - the head proposes the next few tokens at near-zero cost and the trunk verifies them in one pass. Output is byte-for-byte the trunk's own (exact-match verification); the head only changes speed. It is the smallest self-drafting volume in this org, and the cheapest way to stand up a chat deployment that still decodes fast.

It does not replace qwen3.5-0.8b-gguf, and the two are not interchangeable. That volume exists to be a draft for qwen3.5-9b-gguf, qwen3.6-27b-gguf and qwen3.5-122b-gguf; in that role a draft model's own MTP head is never consulted (the target does the verifying), so attaching this volume as a draft buys nothing and costs the extra head's weights. Use this one when the 0.8B is the model being served, and the plain one when it is drafting for something bigger.

Upstream unsloth/Qwen3.5-0.8B-MTP-GGUF ships twenty-one quantizations plus three vision mmproj files and no tokenizer.json. This volume carries ONLY the one served quant plus the tokenizer (same rationale as the sibling volumes): a multi-gguf volume makes the host's preload pick ambiguous, and the extra files never serve.

Q8_0 rather than a 4-bit quant because at 0.8B the whole volume is under a gigabyte either way - UD-Q4_K_XL would save ~0.26 GB, which is noise against any share this model would be deployed on, and 4 bits is where quantization damage starts to show on a model this small. It matches the Q8_0 the sibling qwen3.5-0.8b-gguf serves.

Note the upstream file name is the same one the non-MTP repo uses (Qwen3.5-0.8B-Q8_0.gguf) but the bytes differ - this one is ~22 MB larger and hashes differently, because it carries the extra head. Volumes mount at /models/<volume-name>/ so the two never collide in a guest, but do not assume a file of that name is the plain build; check the sha256 below.

Provenance

The tokenizer is byte-identical to the ones in qwen3.5-0.8b-gguf, qwen3.5-9b-gguf, qwen3.6-27b-gguf, qwen3.6-27b-mtp-gguf and qwen3.5-122b-gguf (the Qwen3.5/3.6 family shares it - same sha256, vocab 248320, <|im_end|> 248046 / <|endoftext|> 248044), which is exactly the property the speculative-decoding compatibility gate requires.

Model geometry (GGUF arch qwen35, hybrid-SSM trunk + 1 dense-attention MTP layer): 24 trunk layers, full_attention_interval 4 (6 trunk KV layers), 2 KV heads, head_dim 256, trained context 262144. For VRAM budgeting count the MTP head as one more full-attention KV layer (7 total) - its context is created lazily when a speculative session first opens. The GGUF reports block_count 25 and nextn_predict_layers 1; the head is blk.24, carrying the nextn.eh_proj / enorm / hnorm / shared_head_norm tensors.

The base model is multimodal upstream (image-text-to-text) and its projector is a qwen3vl_merger, but this volume is TEXT-ONLY: no mmproj is bundled and the catalog entry does not set vision. qwen3-vl-8b-gguf is the vision volume in this org.

LICENSE is the Apache-2.0 text; both upstreams are Apache-2.0.

Usage on Enclave

Wrapped as a Tinfoil Modelwrap volume (dm-verity; the root hash is part of the enclave measurement). Deployments attach it by name; the wasm-manager mounts it read-only at /models/qwen3.5-0.8b-mtp-gguf.