Downloads · 30 days
1.2K
5% of all-time downloads
AesSedai/Qwen3.6-35B-A3B-GGUF
Qwen3.6-35B-A3B-GGUF is a machine learning model from AesSedai. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
- 05-18-26: - ~~I have re-uploaded all quants with MTP tensors (@ Q80) now included.~~ - ~~Edit: I forgot to do --fuse-gate-up-exp so the quants are missing imatrix for some weights. Redoing and re-uploading. Will upd…
Downloads · 30 days
1.2K
5% of all-time downloads
All-time downloads
25.8K
Public
Repo size
118 GB
Likes
32
Public
Click a slice to open those files.
.gguf118 GB · 100%
From the Hugging Face model README
--fuse-gate-up-exp so the quants are missing imatrix for some weights. Redoing and re-uploading. Will update when complete.This repo contains specialized MoE-quants for Qwen3.6-35B-A3B. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. To that end, the quantization type default is kept in high quality and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors.
| Quant | Size | Mixture | PPL | 1-(Mean PPL(Q)/PPL(base)) | KLD |
|---|---|---|---|---|---|
| Q8_0 | 35.19 GiB (8.51 BPW) | Q8_0 | 6.722728 ± 0.043683 | +0.0412% | 0.005988 ± 0.000117 |
| Q6_K | 27.93 GiB (6.76 BPW) | Q8_0 / Q6_K / Q6_K / Q6_K | 6.719363 ± 0.043660 | -0.0089% | 0.006522 ± 0.000094 |
| Q5_K_M | 25.28 GiB (6.12 BPW) | Q8_0 / Q5_K / Q5_K / Q6_K | 6.731999 ± 0.043789 | +0.1791% | 0.008319 ± 0.000134 |
| Q4_K_M | 21.45 GiB (5.19 BPW) | Q8_0 / Q4_K / Q4_K / Q5_K | 6.742029 ± 0.043830 | +0.3284% | 0.013652 ± 0.000131 |
| IQ4_XS | 17.23 GiB (4.17 BPW) | Q8_0 / IQ3_S / IQ3_S / IQ4_XS | 6.878696 ± 0.044917 | +2.3622% | 0.032913 ± 0.000260 |
| IQ3_S | 13.48 GiB (3.26 BPW) | Q6_K / IQ2_S / IQ2_S / IQ3_S | 7.168835 ± 0.047360 | +6.6797% | 0.084335 ± 0.000565 |
