Qwen3.6-35B-A3B-GGUF

Qwen3.6-35B-A3B — cafkafk Q4_K_M with Q8 vision and MTP

GGUF conversions and quantization of Qwen/Qwen3.6-35B-A3B at source revision 995ad96eacd98c81ed38be0c5b274b04031597b0, prepared by cafkafk. Original model and weights by Qwen. The original Apache-2.0 license accompanies these files. This is a quantized derivative, not a fine-tune.

SHA-256:

42aedc12d65b18d188edb5979b6c55803fb28cce80e126689448e43400f95441  Qwen3.6-35B-A3B-Q4_K_M.gguf
3203e07f04f6d0208e81492cd0ffd34517f27d6104f24185e89b1de13c4133c1  mmproj-Qwen3.6-35B-A3B-Q8_0.gguf
93486a5838c7e3418b81f4b727b0f4ff3c7dca2f0759cfaf3dab47f3d56febd3  Qwen3.6-35B-A3B-MTP-Q8_0.gguf

The main model uses the pinned mainline llama.cpp Q4_K_M preset without an importance matrix. The MTP companion uses Q8_0. Vision uses the converter's Q8_0 path, including its F16 fallback for non-block-aligned FFN tensors. Q8_0 therefore describes the chosen conversion mode, not every tensor's type. Original tokenizer, chat template and context/RoPE metadata are retained. GGUF general.name is Original; the filename identifies the release.

The text/vision pair passed CPU inference checks with mainline llama.cpp build 10273 (a6aa6f5) and the retained ik-llama.cpp binary. Each passed arithmetic, PNG color identification, tools, vision with tools, tool-result consumption and sequence generation. The ik check also loaded the MTP companion and reported 96 drafted / 96 accepted tokens on this small fixture. That is not a general acceptance-rate or speed benchmark. The mainline check did not exercise MTP. The exact tested engine builds and flags were recorded at build time but are not published here; compatibility with arbitrary older builds is not implied.

Initial WikiText test-slice perplexities (four 512-token chunks, CPU, same pinned evaluator and corpus):

The downloaded comparison is revision baec3ebee244827cda0f4557eafa8b28f7545fa6 of ggml-org/Qwen3.6-35B-A3B-GGUF; its text quant uses a different tensor precision mix. The small sample does not establish a general quality ranking. The BF16 and new Q4 runs share a host; the downloaded baseline ran on a different host, so these measurements should not be used to compare throughput.

The capability fixture used context 8192 and disabled thinking. Full 262144-token context, thinking quality and GPU behavior have not been validated by this run. The test corpus was not used for quant calibration; no claim is made about the original model's training-data overlap.

Usage

Text only:

llama-server -m Qwen3.6-35B-A3B-Q4_K_M.gguf

For images, supply the vision file through the engine's --mmproj option:

llama-server -m Qwen3.6-35B-A3B-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-Q8_0.gguf \
  --ctx-size 8192

The MTP file is optional and requires an engine with this model's MTP support. In the tested ik build, the additional arguments for CPU drafting were:

llama-server -m Qwen3.6-35B-A3B-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-Q8_0.gguf \
  --model-draft Qwen3.6-35B-A3B-MTP-Q8_0.gguf -ngld 0 \
  --spec-type mtp:n_max=2,p_min=0.0

These are the tested experiment settings, not recommended universal performance defaults.

Verify downloads against the published checksums:

sha256sum -c --ignore-missing SHA256SUMS