Downloads · 30 days
28
1% of all-time downloads
localweights/Qwen3.6-35B-A3B-MTP-IMAT-Q4_K_M-Q8nextn-GGUF
Qwen3.6-35B-A3B-MTP-IMAT-Q4_K_M-Q8nextn-GGUF is a machine learning model from localweights. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
bash llama-cli -m Qwen3.6-35B-A3B-MTP-IMAT-Q4KM-Q8nextn.gguf -ngl 999 -fa on -c 32768 \ --temp 0.7 -p "Generate a short poem about sunrise."
Downloads · 30 days
28
1% of all-time downloads
All-time downloads
2.9K
Public
Repo size
21.7 GB
Likes
3
Public
Click a slice to open those files.
.gguf21.7 GB · 100%
From the Hugging Face model README
---
license: apache-2.0
base_model: Qwen/Qwen3.6-35B-A3B
tags:
- qwen
- 3.6
- 35b
- a3b
- mtp
- gguf
- quantized
pipeline_tag: text-generation
---
# Qwen3.6‑35B‑A3B‑MTP‑IMAT‑Q4_K_M‑Q8nextn (GGUF)
MoE 35‑billion‑parameter Qwen‑3.6‑A3B model quantized with an
imatrix‑calibrated Q4_K_M body and a Q8_0 NextN head.
Offers a higher‑quality body quantization than IQ4_XS at the cost of a larger file.
**File size:** ~20.7 GB (`Qwen3.6-35B-A3B-MTP-IMAT-Q4_K_M-Q8nextn.gguf`)
## Quantization details
- **Body:** Q4_K_M (imatrix‑calibrated)
- **Head:** Q8_0 (NextN) – retains draft fidelity for MoE routing
- **MTP support:** Requires llama.cpp compiled with Qwen 3.5/3.6 MTP.
## Quick‑start with llama.cpp
```bash
llama-cli -m Qwen3.6-35B-A3B-MTP-IMAT-Q4_K_M-Q8nextn.gguf -ngl 999 -fa on -c 32768 \
--temp 0.7 -p "Generate a short poem about sunrise."
-ngl if GPU memory is limited.-fa on enables Flash‑Attention for the MTP pipeline.-c as needed.localweights/Qwen3.6-35B-A3B-MTP-BF16-GGUFlocalweights/Qwen3.6-35B-A3B-MTP-IMAT-IQ4_XS-Q8nextn-GGUF© 2024 The Qwen Team. Licensed under the Apache License, Version 2.0.