Qwen3.8-test-Q4Q2

Qwen3.8-test-Q4Q2 — mixed Q4/Q2 FFN quantization (emulated, GGUF)

Experimental. A 50/50 weight merge of Qwen3.6-27B and Qwen3.8-27B, quantized to llama.cpp Q4_0, with a mixed-precision emulation applied to the FFN:

• per layer, the 25% most drift-stable FFN channels (by min cosine of gate/up/down rows between Qwen3.6 and Qwen3.8) keep their full 4-bit codes;
• the remaining 75% of channels are reduced to 2 effective bits: their Q4_0 codes are rounded to the nearest of {0, 4, 8, 12} (xy00), block scales unchanged.

This is an emulation of a Q4/Q2 mixed format: the file is a standard Q4_0 GGUF (same size, ~15.5 GB); only the information content of the unstable channels is Q2. A native split-tensor implementation would land at ~3.6 bits/weight overall (~12.3 GB).

The chat template is Qwen3.8's. The MTP head is included (nextn layer, kept at full Q4_0) — enable self-drafting speculative decoding with:

llama-server -m Qwen3.8-test-Q4Q2.gguf --spec-type draft-mtp

Observed on an Apple Silicon Mac (Metal): smoke tests indistinguishable from the plain Q4_0 baseline at T=0; 46–48 tok/s decode at 65–79% MTP draft acceptance (mean accepted chain ~3.3).

Give generation enough budget: the template enables thinking, which can consume several hundred tokens before the answer.