Downloads · 30 days
983
8% of all-time downloads
Qwen3.8-Flash-Next-GGUF
vcruz305/Qwen3.8-Flash-Next-GGUF
Qwen3.8-Flash-Next-GGUF is a machine learning model from vcruz305. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
Real K-quant ladder from full HF BF16 weights with MTP heads in-main (blk.48 / nextn.) and a shared BF16 PLE.
All-time downloads
12.5K
Public
Repo size
538 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.gguf538 GB · 100%
At a glance
- License
- other
- Access
- Public
- Created
- Aug 25, 2026
- Updated
- Sep 10, 2026
- SHA
- 0c50dde2
Base models
Model card
From the Hugging Face model README
Qwen3.8-Flash-Next GGUF (vcruz305)
Real K-quant ladder from full HF BF16 weights with MTP heads in-main (blk.48 / nextn.*) and a shared BF16 PLE.
Layout
| path | content |
|---|---|
Q2_K/ … Q6_K/ | Backbone shards 00001–00006 (MTP included in tensor set) |
PLE-BF16/Qwen3.8-Flash-Next-PLE-BF16.gguf | Shared BF16 PLE (~95.4 GiB) — use as each quant's 00007 |
imatrix/ | Importance matrix used for Q2–Q6 |
scripts/link-ple.sh | ln/cp PLE → …-00007-of-00007.gguf per quant |
MTP/ | Optional standalone MTP draft heads (if present) |
Setup
bash scripts/link-ple.sh
Approximate backbone sizes (GiB) + shared PLE
| quant | backbone | MTP tensors |
|---|---|---|
| Q2_K | ~49.5 | ≥20 |
| Q3_K_M | ~62.5 | ≥20 |
| Q4_K_M | ~80.2 | ≥20 |
| Q5_K_M | ~91.3 | ≥20 |
| Q6_K | ~107.8 | ≥20 |
| PLE BF16 (shared) | ~95.4 | — |
Run (DGX Spark / unified memory)
Requires qwen4exp-capable llama.cpp (PR #27742 class).
llama-cli \
-m Q4_K_M/Qwen3.8-Flash-Next-Q4_K_M-00001-of-00007.gguf \
--load-mode mmap \
-ngl 99 \
-ot "per_layer_token_embd.weight=CPU" \
-c 1024 -n 64 -st --temp 0 \
-p "The capital of France is"
Do not mlock the PLE on 128G unified-memory boxes. PLE stays NVMe-backed via mmap.
MTP speculative decode
Weights include MTP (nextn_predict_layers / blk.48). Runtime --spec-type draft-mtp needs a llama.cpp build with qwen4exp graph_mtp (not all mainline builds yet).
Build notes
- Converter: llama.cpp qwen4exp with MTP export enabled
- Imatrix: AtomicChat-compatible matrix
- PLE left BF16 (not re-quantized); tok emb Q8_0; output Q6_K
- License
- other
- Created
- Aug 25, 2026
- Updated
- Sep 10, 2026