Downloads · 30 days
1.8K
15% of all-time downloads
liodon-ai/Qwen3.6-35B-A3B-imatrix-GGUF
Qwen3.6-35B-A3B-imatrix-GGUF is a text generation model from liodon-ai. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
Higher-quality GGUF quantizations of Qwen/Qwen3.6-35B-A3B using importance matrix (iMatrix) calibration.
Downloads · 30 days
1.8K
15% of all-time downloads
All-time downloads
12K
Public
Repo size
282 GB
Likes
1
Trending 1
Click a slice to open those files.
.gguf141 GB · 100%
From the Hugging Face model README
Higher-quality GGUF quantizations of Qwen/Qwen3.6-35B-A3B using importance matrix (iMatrix) calibration.
Standard quantization rounds all weights equally. iMatrix runs a calibration pass over real text to identify which weights matter most, then prioritizes precision where it counts. The result: noticeably better coherence and instruction-following at Q2/Q3/Q4 — same file size, better output.
The i-quants (IQ2_M, IQ3_M, IQ4_XS) are exclusively iMatrix-based and provide the best quality-per-GB available.
ollama run hf.co/liodon-ai/Qwen3.6-35B-A3B-imatrix-GGUF:Q4_K_M
llama-cli -hf liodon-ai/Qwen3.6-35B-A3B-imatrix-GGUF:Q4_K_M
Search liodon-ai/Qwen3.6-35B-A3B-imatrix-GGUF and pick your quant.
| Quant | Size | Notes |
|---|---|---|
Q2_K | 12.94 GB | tiniest standard — runs almost anywhere |
Q3_K_M | 16.76 GB | great for 8GB VRAM |
Q4_K_M | 21.17 GB | sweet spot (recommended) |
Q5_K_M | 24.73 GB | high quality |
Q6_K | 28.51 GB | near-lossless |
Q8_0 | 36.90 GB | basically full quality |
| VRAM | Recommended Quant |
|---|---|
| 6 GB | IQ2_M |
| 8 GB | IQ3_M or Q3_K_M |
| 10 GB | IQ4_XS or Q4_K_M |
| 12 GB | Q4_K_M |
| 16 GB | Q5_K_M |
| 24 GB | Q6_K or Q8_0 |
At low bit widths (Q2/Q3/Q4), standard quantization loses coherence and starts producing repetitive or broken output. iMatrix keeps the model sharp by protecting the most important weights. If you're running at Q4 or below, prefer the iMatrix quants from this repo over standard Q-series from other repos.