Downloads · 30 days
550
18% of all-time downloads
Qwen3.8-27B-NVFP4-GGUF
Siri0s/Qwen3.8-27B-NVFP4-GGUF
Qwen3.8-27B-NVFP4-GGUF is a image-text-to-text model from Siri0s. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as apache-2.0.
Conversion of the pre-quantized checkpoint nvidia/Qwen3.8-27B-NVFP4 (Alibaba Qwen3.8-27B, quantized with NVIDIA Model Optimizer, Apache 2.0 license) to llama.cpp GGUF format while preserving the original quantizations.
All-time downloads
3K
Public
Repo size
22.4 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.gguf22.4 GB · 100%
At a glance
- Task
- Image-Text-to-Text
- Library
- llama.cpp
- License
- apache-2.0
- Access
- Public
- Created
- Sep 16, 2026
- Updated
- Sep 23, 2026
- SHA
- f995028e
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3.8 27B NVFP4 → GGUF as is (except FP8)
Conversion of the pre-quantized checkpoint nvidia/Qwen3.8-27B-NVFP4 (Alibaba Qwen3.8-27B, quantized with NVIDIA Model Optimizer, Apache 2.0 license) to llama.cpp GGUF format while preserving the original quantizations.
Multimodal model: text + images + video. The checkpoint config declares the architecture as Qwen3_5ForConditionalGeneration (model_type qwen3_5) — that is how llama.cpp recognizes it (GGUF arch qwen35).
Files
| File | Size | Contents |
|---|---|---|
qwen38-27b-nvfp4.gguf | 21.5 GB | Language model (64 layers, hidden 5120, vocab 248320) |
mmproj-qwen38-bf16.gguf | 931 MB | Vision encoder (ViT, 27 layers), BF16 |
What it was
A mixed-precision checkpoint from NVIDIA Model Optimizer (hf_quant_config.json, quant_algo: MIXED_PRECISION):
| Quant | Coverage | Tensors | Format in safetensors |
|---|---|---|---|
| NVFP4 (group 16) | all MLPs (gate/up/down × 64 layers) + lm_head | 193 | U8 (nibble-packed) + F8_E4M3 group scales + F32 weight_scale_2 + F32 input_scale |
| FP8 E4M3 | attention: linear_attn (in_proj_qkv/in_proj_z/out_proj × 48) + self_attn (q/k/v/o × 16) | 208 | F8_E4M3 weights + per-tensor F32 weight_scale + input_scale |
| BF16 | embed_tokens, norms, conv1d, A_log, dt_bias, output norm, entire vision tower | ~798 | BF16 |
Architecture: 64 layers, hybrid attention — 48 linear-attention (gated delta net) layers + 16 full-attention layers (every 4th layer).
What it became
| Source quant | Result in GGUF | How |
|---|---|---|
| NVFP4 | NVFP4 (ggml block_nvfp4) | Bit-exact lossless repack into ggml super-blocks (36 bytes per 64 elements: 4 UE4M3 scales + 32 code bytes); original E4M3 scale bits preserved; scale_2/input_scale kept as sidecar tensors and applied at inference via ggml_mul |
| FP8 E4M3 | Q8_0 | Dequantized to F32 and written directly as Q8_0 (--fp8-as-q8) — the only non-lossless step |
| BF16 | BF16 (norms and 1D tensors → F32) | Unchanged |
Resulting file: arch qwen35, ftype MOSTLY_NVFP4 (39), 1252 tensors: NVFP4 — 193, Q8_0 — 208, F32 — 746, BF16 — 105.
The vision tower is a separate mmproj file (standard llama.cpp split), fully BF16.
Run example on RTX-5090-32Gb for Coding (~100 tps)
# cmd
llama-server.exe ^
--model qwen38-27b-nvfp4.gguf ^
--flash-attn on ^
-lm none ^
--jinja ^
--fit on ^
--ctx-size 154000 ^
--temp 0.6 ^
--min-p 0.00 ^
--top-p 0.95 ^
--top-k 20 ^
--presence-penalty 0.05 ^
--repeat-penalty 1.05 ^
--host 0.0.0.0 ^
--port 8082 ^
--reasoning on ^
--spec-type draft-mtp ^
--spec-draft-n-max 3 ^
--reasoning-budget 15000 ^
--reasoning-effort medium
- Task
- Image-Text-to-Text
- Library
- llama.cpp
- License
- apache-2.0
- Created
- Sep 16, 2026
- Updated
- Sep 23, 2026