Downloads · 30 days
186
9% of all-time downloads
AesSedai/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16-GGUF
NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16-GGUF is a machine learning model from AesSedai. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This repo contains specialized MoE-quants for NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible…
Downloads · 30 days
186
9% of all-time downloads
All-time downloads
2.1K
Public
Repo size
1.3 TB
Likes
3
Public
Click a slice to open those files.
.gguf1.3 TB · 100%
From the Hugging Face model README
This repo contains specialized MoE-quants for NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. To that end, the quantization type default is kept in high quality and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors.
| Quant | Size | Mixture | PPL | 1-(Mean PPL(Q)/PPL(base)) | KLD |
|---|---|---|---|---|---|
| Q8_0 | 544.13 GiB (8.51 BPW) | Q8_0 | 3.118690 ± 0.016792 | +0.0988% | 0.001246 ± 0.000016 |
| Q5_K_M | 396.00 GiB (6.19 BPW) | Q8_0 / Q5_K / Q5_K / Q6_K | 3.123444 ± 0.016830 | +0.2514% | 0.002382 ± 0.000021 |
| Q4_K_M | 334.13 GiB (5.22 BPW) | Q8_0 / Q4_K / Q4_K / Q5_K | 3.131514 ± 0.016885 | +0.5104% | 0.005548 ± 0.000048 |
| IQ4_XS | 258.60 GiB (4.04 BPW) | Q6_K / IQ3_S / IQ3_S / IQ4_XS | 3.146202 ± 0.016926 | +0.9819% | 0.016777 ± 0.000139 |
| IQ3_S | 207.98 GiB (3.25 BPW) | Q6_K / IQ2_S / IQ2_S / IQ3_S | 3.239389 ± 0.017506 | +3.9728% | 0.052122 ± 0.000424 |
