Downloads · 30 days
156
4% of all-time downloads
FreedomAISVR/Qwen3.6-35B-A3B-MXFP4-MOE-Fast-GGUF
Qwen3.6-35B-A3B-MXFP4-MOE-Fast-GGUF is a image-text-to-text model from FreedomAISVR. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
GGUF MXFP4 MoE quantization of unsloth/Qwen3.6-35B-A3B-NVFP4-Fast, a 35B parameter MoE model with 3B active parameters.
Downloads · 30 days
156
4% of all-time downloads
All-time downloads
3.5K
Public
Repo size
21.2 GB
Likes
8
Public
Click a slice to open those files.
.gguf21.2 GB · 100%
From the Hugging Face model README
GGUF MXFP4 MoE quantization of unsloth/Qwen3.6-35B-A3B-NVFP4-Fast, a 35B parameter MoE model with 3B active parameters.
Unsloth's NVFP4 Fast variant is a speed-optimized quantization that delivers 1.79x faster throughput than other NVFP4 quants. This GGUF extends that optimization to MXFP4 MoE format:
| Variant | MMLU-Pro | GPQA | AIME 2025 |
|---|---|---|---|
| Unsloth NVFP4 Fast | 85.58 | 87.75 | 91.67 |
| Unsloth NVFP4 | 85.85 | 86.74 | 92.29 |
| NVIDIA NVFP4 | 85.60 | 87.12 | 91.88 |
| BF16 | 85.75 | 86.36 | 92.50 |
This quantization uses a hybrid approach for optimal quality:
MXFP4 is an open standard supported by AMD, NVIDIA, and Microsoft, making it compatible with a wider range of hardware.
Qwen3.6-35B-A3B is a multimodal MoE model from Alibaba's Qwen team:
| File | Size | Description |
|---|---|---|
qwen36-35b-a3b-fast-mxfp4_moe.gguf | ~18.9 GB | MXFP4 MoE quantized text model |
mmproj-qwen36-35b-a3b-f16.gguf | ~0.84 GB | Vision encoder (F16) |
llama-server \
-m qwen36-35b-a3b-fast-mxfp4_moe.gguf \
--mmproj mmproj-qwen36-35b-a3b-f16.gguf \
-ngl 99 \
--host 0.0.0.0 \
--port 8080
Quantized from Qwen/Qwen3.6-35B-A3B BF16 weights using llama.cpp (llama-quantize.exe --allow-requantize MXFP4_MOE).
Apache 2.0 - same as the base model.