Qwopus3.8-27B-Flash-NVFP4

⚡ Qwopus3.8-27B-Flash NVFP4

Follow me on X @BennyDaBall_OG !

A native NVFP4 GGUF of Jackrong/Qwopus3.8-27B-Flash, built for Blackwell GPUs without tossing the fun parts overboard. The dense text backbone is NVIDIA 4-bit floating point, the complete NextN/MTP speculative head stays intact, and the matching BF16 vision projector comes along for the ride.

This is a fresh quantization from the pinned BF16 checkpoint. It is not a requantization of another GGUF. No retraining, no distillation, no mystery meat: just a careful conversion, an explicit precision map, and a frankly unreasonable amount of checking afterward.

---

🧠 What is this?

• 27B Qwopus Flash fine-tune, native NVFP4 backbone. The large attention, Gated DeltaNet, and MLP matrices across all 64 text layers use GGML tensor type 40 (nvfp4).
• MTP retained at high precision. The GGUF has 65 blocks. blk.64 is the embedded MTP layer, so a compatible llama.cpp build can self-speculate with --spec-type draft-mtp and no external draft model.
• BF16 output, embeddings, and large MTP matrices. Small state, convolution, bias, norm, and scale tensors remain F32.
• Exact-source vision. The separate BF16 mmproj was converted from the same pinned Qwopus checkpoint. It was not borrowed from another Qwen3.8 fine-tune.
• 262,144 native context.
• LM Studio / llama.cpp chat template included. The tested template is embedded in the GGUF and also shipped as chat_template.jinja. It supports multimodal content, tool calls, merged system/developer messages, later system messages, and preserved reasoning content.

Use it when you want Qwopus Flash as a fast local coding/agent model on one 5090, with built-in speculative decoding and image input still alive. Tiny file? No. Much smaller than 51.8 GiB of BF16 while keeping the expensive bits where they matter? Absolutely.

---

📦 The files

d39d8abb3bff9074080b94513a2a55a2ff60e69759f586c8e5387804d9e11eeb  *Qwopus3.8-27B-Flash-NVFP4.gguf
e42bbc2ba6928c8ea459cf5d5a958c00866ed072bfd174f050ed497b6ad220e4  *mmproj-Qwopus3.8-27B-Flash-BF16.gguf
e5721ffcc1eec9268925401907b19d945b4419a0abf616013b90ed4ee671a8e8  *chat_template.jinja
3e85d5a338133e9c975da92c009cf3bbcbb42557fbafc45fc33c9dc3e537ba240  *Qwen3.8-27B-NVFP4-imatrix.dat
dee4a09bc0965f914db5b501c42797ecb987d0ba614142e4bca842ea8a6e97c9  *Qwopus3.8-27B-Flash-NVFP4-tensor-types.txt

Text-only use does not need the projector. Add the mmproj when you need image input.

---

🛠️ Requirements

• Built, optimized, and measured for one 32 GB RTX 5090 (sm_120). Other Blackwell cards may work, but they were not the validation target here.
• A recent llama.cpp build with qwen35, NVFP4 CUDA kernels, multimodal support, and the draft-mtp speculative path.
• Q8_0 KV cache is the tested quality and memory balance for this release.

NVFP4 support is runtime-specific. An older llama.cpp or LM Studio runtime may fail to load type-40 tensors even when it supports ordinary Qwen3.8 GGUFs.

---

🚀 Quick start

Text and embedded MTP

llama-server \
  --model Qwopus3.8-27B-Flash-NVFP4.gguf \
  --no-mmproj \
  --ctx-size 262144 \
  --flash-attn on \
  -ctk q8_0 -ctv q8_0 \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --spec-draft-p-split 0.2 \
  --jinja \
  --reasoning-effort xhigh \
  --reasoning-preserve \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 \
  --presence-penalty 0.0 --repeat-penalty 1.0

Vision

llama-server \
  --model Qwopus3.8-27B-Flash-NVFP4.gguf \
  --mmproj mmproj-Qwopus3.8-27B-Flash-BF16.gguf \
  --ctx-size 131072 \
  --flash-attn on \
  -ctk q8_0 -ctv q8_0 \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --spec-draft-p-split 0.2 \
  --jinja \
  --chat-template-file chat_template.jinja \
  --reasoning-effort xhigh \
  --reasoning-preserve \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 \
  --presence-penalty 0.0 --repeat-penalty 1.0

The vision profile uses a smaller context allocation to leave headroom for image encoding on a 32 GB RTX 5090. Treat context and batch settings as allocations, not a promise that every prompt length fits every desktop state.

LM Studio

Download Qwopus3.8-27B-Flash-NVFP4.gguf and load it normally. Add the matching BF16 projector only when you need image input. LM Studio should read the tested chat template directly from the GGUF, so a separate template import is normally not required; chat_template.jinja is included for runtimes or presets that need an explicit override.

For embedded MTP, start with maximum draft tokens 3 and minimum continue probability 0.2. Q8_0 K/V cache is the measured setting for this release.

Thinking defaults and output headroom

This fine-tune inherits Qwen3.8's chat behavior. For thinking mode, the official Qwen3.8-27B model card recommends temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, and repetition_penalty=1.0. It identifies xhigh as the default reasoning effort and enables preserved thinking by default.

Do not give a difficult agent or coding task a tiny output allowance and then mistake a truncated reasoning trace for a model failure. Qwen's own QwenSWEBench setup used a 32,768-token output limit with a 256K context window; for long-horizon local work, use similarly generous headroom or leave prediction uncapped when the runtime and workflow can tolerate it.

One local no-cap probe of this NVFP4 artifact used those official thinking-mode sampling defaults, xhigh, 229,376 allocated context, and no reasoning or completion cap. It stopped naturally after 2,206 completion tokens in 19.9 seconds at 155.1 tok/s. Its final Python had malformed indentation and failed the executable gate. That is included here deliberately: measured throughput does not imply answer quality, and a single sample is not an evaluation.

---

📈 Measured on RTX 5090

Test system: RTX 5090 32 GB, one server slot, flash attention on, Q8_0 K/V cache, 131,072 allocated context, deterministic paired prompts, and the local llama.cpp NVFP4/MTP build.

Native MTP n=3 raised mean decode speed from 72.67 to 114.42 tok/s, a 57.5% increase, while remaining self-contained in the main GGUF. MTP n=2 is a useful higher-acceptance alternative: it measured 112.30 tok/s mean with 69.0% weighted acceptance. The external DFlash2 row is the fastest optional text lane, but its draft model is not included in this repository.

The source model card reports its own MTP measurements on a Q5_K_M target. The table above is a separate measurement of this NVFP4 artifact on one RTX 5090. Acceptance varies with prompts, sampling, runtime revision, and draft depth.

---

🔬 Tensor receipt

• Main GGUF: 866 tensors, 65 blocks, one NextN layer.
• Precision layout: 448 NVFP4, 408 F32, 10 BF16 tensors.
• BF16 tensors: output head, token embeddings, and the eight large block-64 MTP matrices.
• Projector: 334 tensors, with 110 BF16 and 224 F32 tensors.
• Source checkpoint: pinned at 44d24e8cb20ceb3cdf4fe200b5a0afd970ee748a.

The source index and all downloaded shards were checked independently before conversion. All 1,199 source tensors were BF16, including all 15 mtp.* tensors and all 333 vision-tower tensors.

---

🧬 The conversion recipe

I believe in open recipes. The two small calibration files used for this build are included in the repo, so the precision layout is inspectable instead of being a hand-wave and a filename.

The build used the MTP-aware Qwen3.8 converter and quantizer from llama.cpp commit 5ecbe1ac17ec0484c5b44af0bd580cdc9c428ed4. In abbreviated shell form:

SOURCE=Qwopus3.8-27B-Flash-BF16
PARENT=Qwopus3.8-27B-Flash-BF16-MTP.gguf
FINAL=Qwopus3.8-27B-Flash-NVFP4.gguf
MATRIX=Qwen3.8-27B-NVFP4-imatrix.dat
TYPES=Qwopus3.8-27B-Flash-NVFP4-tensor-types.txt

hf download Jackrong/Qwopus3.8-27B-Flash \
  --revision 44d24e8cb20ceb3cdf4fe200b5a0afd970ee748a \
  --local-dir "$SOURCE"

python convert_hf_to_gguf.py "$SOURCE" \
  --outfile "$PARENT" \
  --outtype bf16 \
  --model-name "Qwopus3.8 27B Flash BF16 MTP"

llama-quantize --dry-run \
  --imatrix "$MATRIX" \
  --tensor-type-file "$TYPES" \
  "$PARENT" "$FINAL" q8_0 32

llama-quantize \
  --imatrix "$MATRIX" \
  --tensor-type-file "$TYPES" \
  "$PARENT" "$FINAL" q8_0 32

python convert_hf_to_gguf.py "$SOURCE" \
  --outfile mmproj-Qwopus3.8-27B-Flash-BF16.gguf \
  --outtype bf16 \
  --mmproj \
  --model-name "Qwopus3.8 27B Flash BF16 mmproj"

python gguf_new_metadata.py "$FINAL" "$FINAL.clean" \
  --remove-metadata quantize.imatrix.file \
  --force
mv "$FINAL.clean" "$FINAL"

The q8_0 positional value is only the fallback type: the included 866-line tensor map makes the actual decision for every tensor. It sends 448 large backbone matrices to NVFP4, leaves 408 small state/norm/scale tensors in F32, and keeps the output head, token embeddings, and eight large MTP matrices in BF16. The included importance matrix has 200 chunks and 496 entries; it is architecture-matched Qwen3.8 calibration, not Qwopus-specific calibration. The final metadata-only rewrite removes the build machine's calibration-file pathname from the container; it does not requantize or alter tensor data.

After conversion, the build hard-failed unless it found exactly 866 tensors, 65 blocks, 15 MTP tensors, the 448/408/10 precision split, and the exact ten BF16 tensor names. Then it still had to survive generation, full 262K allocation, native MTP drafting, and real image input. Trust, but make the GPU show receipts.

No weights were trained or fine-tuned in this repository. Model behavior comes from Jackrong's source fine-tune. This repository only changes the storage and inference precision.

---

⚠️ Notes

• Native NVFP4 is intended for Blackwell. CPU and non-Blackwell fallback behavior depends on the runtime and is not the target of this build.
• The model thinks by default. In llama-server requests, use chat_template_kwargs: {"enable_thinking": false} when you need a concise, non-thinking response.
• Keep repeat penalty at 1.0 for the tested Qwen3.8 agent profile.

---

🙏 Acknowledgements

• Alibaba / Qwen for Qwen3.8-27B.
• Jackrong for Qwopus3.8-27B-Flash.
• Unsloth and the maintainers behind llama.cpp.

Apache-2.0, matching the upstream model metadata. "Qwen" is a trademark of Alibaba and is used only to identify the upstream model. This repository is not affiliated with or endorsed by Alibaba.

Quantized locally with care and way too many validation passes by BennyDaBall.

Follow me on X @BennyDaBall_OG !