Downloads · 30 days
647
9% of all-time downloads
Jianqiao1/Qwen3.8-27b-MoQ-GGUF
Qwen3.8-27b-MoQ-GGUF is a machine learning model from Jianqiao1. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This repository contains imatrix-aware, layer-aware Mixture-of-Quantization (MoQ) GGUF files for the original Qwen/Qwen3.8-27B model. These files preserve the model's MTP tensors and are quantizations of the base Qwen…
Downloads · 30 days
647
9% of all-time downloads
All-time downloads
7K
Public
Repo size
130 GB
Likes
7
Public
Click a slice to open those files.
.gguf130 GB · 100%
From the Hugging Face model README
This repository contains imatrix-aware, layer-aware Mixture-of-Quantization (MoQ) GGUF files for the original Qwen/Qwen3.8-27B model. These files preserve the model's MTP tensors and are quantizations of the base Qwen model. In terms of overall quality, MoQ is comparable to Unsloth's Dynamic Quantization 3.0; however, our quantization series offers a more granular range of options (4.0 bpw – 4.75 bpw) than the Unsloth team's offerings, while maintaining a slight performance edge over Dynamic Quantization 3.0 within this range.
The complete series was evaluated locally alongside the Unsloth Dynamic GGUF series under identical conditions. Lower is better in all three charts.



The interactive comparison report supports pan, zoom, view reset, and per-series visibility controls. The companion CSV contains all measured values and tensor-composition summaries.
All 17 measured GGUF files contain 866 tensors and 27,320,697,856 parameters. They were evaluated against the same BF16 reference logits on WikiText-2 with a fixed context length of 512.
At the exact size of each of the 9 MoQ files, linearly interpolating the Unsloth curve favors this MoQ series on:
The results show the intended tradeoff clearly. The layer-aware MoQ recipes are especially effective on PPL and tail divergence, while the Unsloth Dynamic recipes remain very strong on Mean KLD, particularly at low and middle file sizes. The full table is included below so that users can choose based on the metric that matters for their workload.
Actual BPW is computed from the complete GGUF file size, including metadata and alignment. GB is decimal. PPL and all KLD values are lower-is-better; same top-p is higher-is-better.
| Series | Recipe | Actual BPW | GB | PPL | Mean KLD | p999 KLD | p99 KLD | RMS delta-p | Same top-p |
|---|---|---|---|---|---|---|---|---|---|
| Jianqiao1 MoQ | 3.2 | 3.0956 | 10.572 | 7.408463 | 0.111637 | 3.348125 | 1.084846 | 9.634% | 85.679% |
| Jianqiao1 MoQ | 3.6 | 3.4821 | 11.892 | 7.221080 | 0.072244 | 2.351531 | 0.717370 | 7.682% | 88.722% |
| Jianqiao1 MoQ | 3.8 | 3.7232 | 12.715 | 7.122706 | 0.048190 | 1.624749 | 0.437946 | 6.077% | 90.109% |
| Jianqiao1 MoQ | 4.1 | 4.1395 | 14.137 | 7.068801 | 0.031460 | 1.056327 | 0.285069 | 4.915% | 91.872% |
| Jianqiao1 MoQ | 4.3 | 4.3600 | 14.890 | 7.045628 | 0.024500 | 0.805452 | 0.216612 | 4.332% | 92.648% |
| Jianqiao1 MoQ | 4.6 | 4.4302 | 15.129 | 7.014436 | 0.020171 | 0.758995 | 0.200817 | 3.936% | 93.809% |
| Jianqiao1 MoQ | 4.8 | 4.6961 | 16.038 | 7.008832 | 0.017282 | 0.650152 | 0.169743 | 3.584% | 94.339% |
| Jianqiao1 MoQ | 4.9 | 4.8055 | 16.411 | 7.021034 | 0.016342 | 0.651529 | 0.162814 | 3.530% | 94.423% |
| Jianqiao1 MoQ | 5.1 | 5.0772 | 17.339 | 7.020549 | 0.012857 | 0.487299 | 0.123491 | 3.134% | 94.968% |
| Unsloth | UD-IQ2_XXS | 2.6383 | 9.010 | 7.957195 | 0.163689 | 4.714475 | 1.739405 | 11.822% | 82.389% |
| Unsloth | UD-IQ2_M | 3.0219 | 10.320 | 7.507406 | 0.103564 | 3.590751 | 1.076759 | 9.332% | 85.496% |
| Unsloth | UD-Q2_K_XL | 3.1263 | 10.676 | 7.403969 | 0.091579 | 3.107871 | 0.925859 | 8.708% | 86.160% |
| Unsloth | UD-IQ3_XXS | 3.4885 | 11.914 | 7.246240 | 0.054517 | 2.259040 | 0.627471 | 6.766% | 90.359% |
| Unsloth | UD-Q3_K_XL | 3.9358 | 13.441 | 7.111811 | 0.031355 | 1.248320 | 0.353498 | 5.129% | 92.396% |
| Unsloth | UD-Q4_K_XL | 5.2483 | 17.923 | 6.976017 | 0.008640 | 0.422880 | 0.090719 | 2.614% | 96.073% |
| Unsloth | UD-Q5_K_XL | 5.9203 | 20.218 | 6.967300 | 0.004526 | 0.200576 | 0.040594 | 1.849% | 97.153% |
| Unsloth | UD-Q6_K_XL | 7.5911 | 25.924 | 6.953912 | 0.001331 | 0.062800 | 0.011536 | 1.107% | 98.493% |
The Unsloth rows are local comparison measurements only. Unsloth GGUF files are not redistributed in this repository.
The recipe number is a series label. Actual BPW below is calculated from the complete file and is the value to use for exact size comparisons.
| File | Actual BPW | Size GB | Size GiB |
|---|---|---|---|
Qwen3.8-27B-MTP-MoQ-3.2.gguf | 3.0956 | 10.572 | 9.846 |
Qwen3.8-27B-MTP-MoQ-3.6.gguf | 3.4821 | 11.892 | 11.075 |
Qwen3.8-27B-MTP-MoQ-3.8.gguf | 3.7232 | 12.715 | 11.842 |
Qwen3.8-27B-MTP-MoQ-4.1.gguf | 4.1395 | 14.137 | 13.166 |
Qwen3.8-27B-MTP-MoQ-4.3.gguf | 4.3600 | 14.890 | 13.867 |
Qwen3.8-27B-MTP-MoQ-4.6.gguf | 4.4302 | 15.129 | 14.090 |
Qwen3.8-27B-MTP-MoQ-4.8.gguf | 4.6961 | 16.038 | 14.936 |
Qwen3.8-27B-MTP-MoQ-4.9.gguf | 4.8055 | 16.411 | 15.284 |
Qwen3.8-27B-MTP-MoQ-5.1.gguf | 5.0772 | 17.339 | 16.148 |
The repository also includes the locally generated calibration imatrix and a BF16 multimodal projector.
This series combines the advantages of the MoQ (Mixed-precision Quantization) scheme at the layer level with tensor-specific strategies derived from Unsloth Dynamic 3.0's low-bit GGUF implementations. Since Qwen3.8 and Qwen3.6 share a matching tensor-level architecture, we were able to transfer and fine-tune layer-specific quantization strategies without being forced to apply a uniform quantization type across all tensor families.
The specific process is as follows:
This is not a single, global quantization preset; different precision settings can be applied to specific tensor families and layers based on their measured or inferred sensitivity.
Compared to the MTP layer in Qwen 3.6, the MTP layer in Qwen 3.8 employs lower-bit quantization; this is because our tests revealed that the prediction performance of Qwen 3.8's MTP layer—even at Q8 precision—falls short of that of Qwen 3.6, rendering the maintenance of high precision for the MTP layer less meaningful.
The comparison used llama.cpp build 10276, commit 6ea215d17, with CUDA on an NVIDIA GeForce RTX 5090. WikiText-2 wiki.test.raw was evaluated against BF16 logits from the original Qwen3.8-27B GGUF. Context length was fixed at 512; logical batch was 2048, ubatch was 8192, GPU layers were selected automatically with fit enabled, op offload and flash attention were enabled, and 16 CPU threads were used. GPU evaluations were run strictly one at a time.
The BF16 reference PPL reported by the evaluator was 6.950282.
Use a recent llama.cpp build with Qwen3.8 and MTP support. The files can be used for ordinary generation or MTP speculative decoding. Choose a file according to available memory and the quality curves above; the nominal recipe label is useful for navigating the series, while the actual file BPW and GB columns provide precise memory-planning values.
Released under the Apache License 2.0, following the base model license metadata.
Thanks to: