Downloads · 30 days
2.1K
11% of all-time downloads
neko-legends/Qwen3.6-35B-A3B-NVFP4-MTP-GGUF
Qwen3.6-35B-A3B-NVFP4-MTP-GGUF is a text generation model from neko-legends. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
<div style="font-family: Inter, ui-sans-serif, system-ui, sans-serif; border: 1px solid 3a2a1f; background: 0f0f13; border-radius: 16px; overflow: hidden; margin: 0 0 28px 0;" <div style="padding: 14px 18px; backgroun…
Downloads · 30 days
2.1K
11% of all-time downloads
All-time downloads
20K
Public
Repo size
72.9 GB
Likes
4
Public
Click a slice to open those files.
.gguf72.9 GB · 100%
From the Hugging Face model README
Native llama.cpp conversions of NVIDIA and Unsloth Qwen3.6 35B A3B NVFP4 checkpoints with their bundled MTP blocks preserved. The newest artifact comes from unsloth/Qwen3.6-35B-A3B-NVFP4-Fast.
| File | Source | Layout | Size |
|---|---|---|---|
qwen3.6-35b-a3b-unsloth-nvfp4-fast-mtp-gguf.gguf | Unsloth NVFP4 Fast | All expert layers native NVFP4, bundled MTP | 21.33 GiB |
qwen3.6-35b-a3b-unsloth-nvfp4-mtp-gguf.gguf | Unsloth NVFP4 | Native NVFP4 expert FFNs, source FP8 tensors stored as Q8_0, bundled MTP | 24.34 GiB |
qwen3.6-35b-a3b-nvfp4-mtp.gguf | NVIDIA NVFP4 | Native NVFP4 with bundled MTP | 22.21 GiB |
Fast GGUF SHA256:
ABE31A8082E44E5C03D0F08595715584AEADEA8C2A76C6F9F066A4ECBA64E78A.
The Fast source stores every expert layer as NVFP4. The earlier Unsloth source
uses a mixed compressed-tensors layout; its conversion keeps packed NVFP4
expert tensors native and stores source FP8 weights as Q8_0. Both expose their
MTP layer to recent llama.cpp builds through draft-mtp.
Windows 11, RTX 5090, 200k context, q4_0 target/draft KV, draft-mtp n=2, no-thinking, identical BookContext fixtures, and one measured 1,024-token
completion per point. These are llama.cpp decode-only rates after the separately
reported prompt prefill.
| GGUF | Runtime | 10k decode | 10k prefill | 200k decode | 200k prefill | VRAM after |
|---|---|---|---|---|---|---|
| Unsloth Fast | b10068 | 135.23 tok/s | 2.16s | 84.08 tok/s | 52.84s | 25,138 MiB |
| Unsloth mixed | b9851 | 151.16 tok/s | 2.37s | 93.20 tok/s | 50.41s | 26.7 GiB |
| NVIDIA source | b9851 | 150.94 tok/s | 2.39s | 90.24 tok/s | 51.35s | 24.6 GiB |
The Fast artifact is about 3.0 GiB smaller than the earlier mixed Unsloth conversion and leaves roughly 7 GiB free with the tested 200k server. However, Unsloth's upstream Fast claim is based on vLLM concurrency throughput. That advantage did not transfer to this native Windows llama.cpp, single-request test: the earlier mixed conversion remains faster at both prompt lengths.
Benchmark CSV: benchmarks/qwen36-35b-a3b-unsloth-nvfp4-fast-mtp-gguf-benchmark-20260719.csv.
llama-server \
--model qwen3.6-35b-a3b-unsloth-nvfp4-fast-mtp-gguf.gguf \
--alias qwen36-35b-a3b-unsloth-nvfp4-fast-mtp-gguf \
--host 0.0.0.0 \
--port 39202 \
--device CUDA0 \
--gpu-layers all \
--gpu-layers-draft all \
--ctx-size 200000 \
--cache-type-k q4_0 \
--cache-type-v q4_0 \
--cache-type-k-draft q4_0 \
--cache-type-v-draft q4_0 \
--flash-attn on \
--parallel 1 \
--cont-batching \
--jinja \
--reasoning off \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
--spec-draft-p-min 0.0
Portable Windows source download, conversion, official llama.cpp b10068 runtime installation, Hermes setup, launcher, and benchmark scripts are in neko-legends/nvidia-local-llm-profiles. Models are stored relative to the checkout parent rather than in a machine-specific drive path.
Apache 2.0, inherited from the source releases.