Downloads · 30 days
835
43% of all-time downloads
groxaxo/Qwen3.8-Flash-Next-Heretic-GGUF
Qwen3.8-Flash-Next-Heretic-GGUF is a text generation model from groxaxo. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as other.
GGUF quantizations of Qwen3.8-Flash-Next-Uncensored (heretic variant), an uncensored/abliterated release of the Qwen3.8 Flash Next sparse MoE architecture.
Downloads · 30 days
835
43% of all-time downloads
All-time downloads
1.9K
Public
Repo size
189 GB
Likes
2
Public
Click a slice to open those files.
.gguf152 GB · 100%
From the Hugging Face model README
GGUF quantizations of Qwen3.8-Flash-Next-Uncensored (heretic variant), an uncensored/abliterated release of the Qwen3.8 Flash Next sparse MoE architecture.
| Folder | Quant | Files | Notes |
|---|---|---|---|
UD-IQ4_XS/ | Unsloth Dynamic IQ4_XS | 41 shards + F16 mmproj | Smallest footprint, full 262K native context |
AD-5.00bpw-Q5_K_M-M64/ | ADQuant 5.00 bpw Q5_K_M-M64 | 32 shards + F16 mmproj | Higher fidelity, M64 expert layout |
Both include the F16 mmproj for vision (image input).
qwen4exp)Because only ~10/512 experts fire per token, the model is far larger on disk than its per-token compute — expect fast decode even on consumer hardware, with expert weights partially served from system RAM via mmap when VRAM is insufficient.
Tuned on 3x RTX 3090 (24 GB) in production:
llama-server -m <model>.gguf --mmproj <mmproj>.gguf \
-c 225280 -ngl 999 -sm layer -ts 1,1,1 \
-fa on -ctk q8_0 -ctv q8_0 \
--jinja --reasoning-preserve \
-md <mtp-draft.gguf> --spec-type draft-mtp --spec-draft-n-max 4 -ngld 999
--load-mode mmap) — sparse activation makes this cheap.Measured on the sibling quant AD-4.27bpw-Q4_K_M-M64 agentic-v2 of the same base model, 225K ctx, Q8 KV, single slot:
| Workload | Decode tok/s | Draft acceptance |
|---|---|---|
| Structured math | 95–98 | 0.80–0.86 |
| Code generation | ~79 | 0.69–0.70 |
| Reasoning / explanation | 56–64 | 0.43–0.53 |
| Creative prose | 52–64 | 0.37–0.53 |
| Vision (mmproj) | ~57 | 0.45–0.48 |
Q5_K_M-M64 shards are a single-file unsplit merge away from the master; use --keep-split workflows if you re-quantize.