Downloads · 30 days
408
11% of all-time downloads
Qwen3.5-0.8B-NVFP4-MTP-GGUF
michaelw9999/Qwen3.5-0.8B-NVFP4-MTP-GGUF
Qwen3.5-0.8B-NVFP4-MTP-GGUF is a machine learning model from michaelw9999. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a GGUF of Qwen3.5-0.8B-NVFP4 with MTP enabled and was quantized with my <Bstill experimental</B <A HREF="https://github.com/michaelw9999/advanced-gguf-quantizer"advanced-gguf-quantizer</A tool.<BR It is faster…
All-time downloads
3.6K
Public
Repo size
1.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf578 MB · 100%
At a glance
- Access
- Public
- Created
- May 25, 2026
- Updated
- Jun 3, 2026
- SHA
- d8ac925b
Base models
Model card
From the Hugging Face model README
This is a GGUF of Qwen3.5-0.8B-NVFP4 with MTP enabled and was quantized with my <B>still experimental</B> <A HREF="https://github.com/michaelw9999/advanced-gguf-quantizer">advanced-gguf-quantizer</A> tool.<BR> It is faster, has better quality, and is smaller than the ModelOpt created version (547MB vs 804MB).<BR> <BR> Recently re-updated with improved quantizer updates and the quality is vastly improved compared to the original version while speed goes up.<BR> This model is very fast on a 5090:<BR>
| qwen35 0.8B NVFP4 | 540.80 MiB | 752.39 M | CUDA | 99 | pp512 | 45086.64 ± 129.78 |
| qwen35 0.8B NVFP4 | 540.80 MiB | 752.39 M | CUDA | 99 | tg128 | 603.17 ± 7.33 t/s |
ModelOpt/converted version:<BR>
| qwen35 0.8B NVFP4 | 792.87 MiB | 772.85 M | CUDA | 99 | pp512 | 41875.80 ± 127.26 |
| qwen35 0.8B NVFP4 | 792.87 MiB | 772.85 M | CUDA | 99 | tg128 | 525.74 ± 3.83 |
<B>Qwen3.5-0.8B-NVFP4-MTP-GGUF:<BR></B>
====== Perplexity statistics ======
Mean PPL(Q) : 21.213287 ± 0.174273
Mean PPL(base) : 18.593291 ± 0.149319
Cor(ln(PPL(Q)), ln(PPL(base))): 98.59%
Mean ln(PPL(Q)/PPL(base)) : 0.131827 ± 0.001377
Mean PPL(Q)/PPL(base) : 1.140911 ± 0.001571
Mean PPL(Q)-PPL(base) : 2.619996 ± 0.036840
====== KL divergence statistics ======
Mean KLD: 0.100037 ± 0.000359
Maximum KLD: 8.239321
99.9% KLD: 1.553173
99.0% KLD: 0.559938
95.0% KLD: 0.279044
90.0% KLD: 0.203463
Median KLD: 0.071593
10.0% KLD: 0.007148
5.0% KLD: 0.002042
1.0% KLD: 0.000229
0.1% KLD: 0.000028
Minimum KLD: -0.000005
====== Token probability statistics ======
Mean Δp: -1.381 ± 0.019 %
Maximum Δp: 63.455%
99.9% Δp: 30.934%
99.0% Δp: 16.788%
95.0% Δp: 8.003%
90.0% Δp: 4.201%
75.0% Δp: 0.377%
Median Δp: -0.070%
25.0% Δp: -2.347%
10.0% Δp: -8.457%
5.0% Δp: -13.786%
1.0% Δp: -28.768%
0.1% Δp: -58.272%
Minimum Δp: -99.590%
RMS Δp : 7.561 ± 0.043 %
Same top p: 83.452 ± 0.097 %
Top flip weight: 0.015182
Top prob RMSE : 0.098831
Entropy RMSE : 0.412400
<B>ModelOpt/converted to GGUF:</B>
====== Perplexity statistics ======
Mean PPL(Q) : 22.587121 ± 0.185301
Mean PPL(base) : 18.593291 ± 0.149319
Cor(ln(PPL(Q)), ln(PPL(base))): 97.53%
Mean ln(PPL(Q)/PPL(base)) : 0.194579 ± 0.001813
Mean PPL(Q)/PPL(base) : 1.214800 ± 0.002203
Mean PPL(Q)-PPL(base) : 3.993830 ± 0.051602
====== KL divergence statistics ======
Mean KLD: 0.172499 ± 0.000603
Maximum KLD: 9.882007
99.9% KLD: 2.801777
99.0% KLD: 0.976789
95.0% KLD: 0.480999
90.0% KLD: 0.349817
Median KLD: 0.122966
10.0% KLD: 0.013817
5.0% KLD: 0.004339
1.0% KLD: 0.000559
0.1% KLD: 0.000083
Minimum KLD: 0.000009
====== Token probability statistics ======
Mean Δp: -2.548 ± 0.025 %
Maximum Δp: 72.600%
99.9% Δp: 35.433%
99.0% Δp: 19.490%
95.0% Δp: 8.787%
90.0% Δp: 4.378%
75.0% Δp: 0.223%
Median Δp: -0.200%
25.0% Δp: -3.814%
10.0% Δp: -12.445%
5.0% Δp: -19.910%
1.0% Δp: -40.716%
0.1% Δp: -75.383%
Minimum Δp: -98.759%
RMS Δp : 10.102 ± 0.051 %
Same top p: 79.130 ± 0.106 %
- Created
- May 25, 2026
- Updated
- Jun 3, 2026