Downloads · 30 days
92
7% of all-time downloads
dawncr0w/Qwen3.8-27B-Heretic-ARA-ModelOpt-NVFP4-GGUF
Qwen3.8-27B-Heretic-ARA-ModelOpt-NVFP4-GGUF is a text generation model from dawncr0w. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
Text-only GGUF conversion of PassingByPixels/Qwen3.8-27B-heretic-ara-NVFP4, paired with a compact Q4KM MTP-only draft model.
Downloads · 30 days
92
7% of all-time downloads
All-time downloads
1.4K
Public
Repo size
20.9 GB
Likes
1
Public
Click a slice to open those files.
.gguf20.9 GB · 100%
From the Hugging Face model README
Text-only GGUF conversion of PassingByPixels/Qwen3.8-27B-heretic-ara-NVFP4, paired with a compact Q4_K_M MTP-only draft model.
Qwen/Qwen3.8-27B
-> trohrbaugh/Qwen3.8-27B-heretic-ara
-> PassingByPixels/Qwen3.8-27B-heretic-ara-NVFP4
-> this GGUF conversion
b10453, --outtype auto --no-nextn.GGML_TYPE_NVFP4).| File | Purpose | SHA-256 |
|---|---|---|
Qwen3.8-27B-Heretic-ARA-ModelOpt-NVFP4.gguf | Target model | 579661c0a28696feae61110f19f6d8509c142f8ee91f01b0f3924dac511513bd |
Qwen3.8-27B-MTP-ONLY-Q4_K_M.gguf | MTP draft model | a120e79f1f517ecb2737005d446649b1d3431864330db2086a26f22b49a2be8d |
The MTP file is derived from the original Qwen3.8-27B MTP head and is compatible with Qwen3.8-27B derivatives. It is not trained on the Heretic weights.
Requires a recent CUDA build with Qwen3.8, NVFP4, and draft-MTP support. Native FP4 execution requires NVIDIA Blackwell.
llama-server \
-m Qwen3.8-27B-Heretic-ARA-ModelOpt-NVFP4.gguf \
-c 262144 -np 1 --kv-unified -ngl all -fa on \
-ctk q8_0 -ctv q4_0 \
--spec-type draft-mtp \
--spec-draft-model Qwen3.8-27B-MTP-ONLY-Q4_K_M.gguf \
--spec-draft-n-max 3 -ctkd q4_0 -ctvd q4_0 \
-b 512 -ub 256 -fit off \
--jinja --reasoning on --reasoning-preserve
Disable thinking per request:
{"chat_template_kwargs":{"enable_thinking":false}}
Tested on one GeForce RTX 5090 32 GB with llama.cpp b10453, CUDA 13.3 runtime, 262,144 allocated context, one slot, Flash Attention, K Q8_0 / V Q4_0 target KV, and Q4_0 draft KV.
| Workload | Target tok/s | MTP accepted/generated | Acceptance | Mean draft length |
|---|---|---|---|---|
| Korean prose, 512 tokens | 72.80 | 269/722 | 37.26% | 2.12 |
| Python code, 512 tokens | 124.39 | 369/423 | 87.23% | 3.62 |
| Repetitive text, 512 tokens | 135.76 | 381/388 | 98.20% | 3.93 |
3 was retained because 5 and 7 reduced throughput on low-acceptance prose.enable_thinking=false, Korean and English output, coding, authentication, start/stop, and VRAM release were tested through the OpenAI-compatible API.These are single-system smoke measurements, not general benchmark claims. Throughput depends heavily on prompt structure, MTP acceptance, context length, runtime revision, and GPU state.
temperature 0.6
top_p 0.95
top_k 20
min_p 0.0
presence_penalty 0.0
repeat_penalty 1.0
Apache License 2.0, inherited from the upstream model lineage.