Downloads · 30 days
64
44% of all-time downloads
Qwen3.8-27B-Heretic-NVFP4
darrellbest/Qwen3.8-27B-Heretic-NVFP4
Qwen3.8-27B-Heretic-NVFP4 is a image-text-to-text model from darrellbest. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
NVFP4 (4-bit) build of darrellbest/Qwen3.8-27B-Heretic, for vLLM on NVIDIA Blackwell, which runs NVFP4 natively. 27 GB instead of 51 GB.
All-time downloads
146
Public
Parameters
19.1B
28.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors28.6 GB · 100%
Parameter types
How the weights are stored.
U818.8B · 68%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3.8-27B Heretic NVFP4
NVFP4 (4-bit) build of darrellbest/Qwen3.8-27B-Heretic, for vLLM on NVIDIA Blackwell, which runs NVFP4 natively. 27 GB instead of 51 GB.
What is quantized
| Part | Precision |
|---|---|
| Linear layers of the MLPs and the 16 full-attention layers | NVFP4, 16-value groups, FP8 scales |
Gated DeltaNet (linear_attn) layers, vision tower, embeddings, lm_head, MTP | bf16 (unchanged) |
This model is 64 layers of which 48 are Gated DeltaNet, and those keep a recurrent state that low precision damages,
so they stay in bf16 along with the vision tower. That is why the file is 27 GB rather than roughly 15 GB.
Multi-token-prediction weights are preserved in model-auxiliary.safetensors, which save_pretrained drops for this
architecture.
Made with llm-compressor 0.13.0 (scheme="NVFP4"), calibrated on
64 harmless chat prompts. Format: compressed-tensors, nvfp4-pack-quantized.
Use
vllm serve darrellbest/Qwen3.8-27B-Heretic-NVFP4
Ablation
0/100 refusals against 98/100 for the original, KL divergence 0.0465. Method and parameters are on the parent card. The 4-bit weights were not re-measured for refusals, and the 4-bit activation path is vLLM's, which was not exercised here.
Reduced safety guardrails by design. You are responsible for what you do with it.
The family
| Repository | Format | Size | Use it with |
|---|---|---|---|
| Qwen3.8-27B-Heretic | bf16 safetensors | 51 GB | transformers, vLLM, SGLang |
| Qwen3.8-27B-Heretic-GGUF | GGUF BF16 / Q8_0 / Q4_K_M (+ vision) | 51 / 27 / 16 GB | llama.cpp, Ollama |
| Qwen3.8-27B-Heretic-FP8 | FP8 W8A8, compressed-tensors | 35 GB | vLLM |
| Qwen3.8-27B-Heretic-NVFP4 | NVFP4, compressed-tensors | 27 GB | vLLM on Blackwell |
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Sep 24, 2026
- Updated
- Sep 24, 2026