Downloads · 30 days
219
3% of all-time downloads
esatapedico/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-GGUF
Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-GGUF is a text generation model from esatapedico. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
A family of four tensor-spliced hybrid GGUFs of Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP:
Downloads · 30 days
219
3% of all-time downloads
All-time downloads
8.5K
Public
Repo size
84.5 GB
Likes
2
Public
Click a slice to open those files.
.gguf69 GB · 100%
From the Hugging Face model README
A family of four tensor-spliced hybrid GGUFs of Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP:
The goal was simple: keep the NVFP4 prefill advantage, shrink the file below the 24 GB budget, and get faster decode via a higher-quality MTP draft head. On our dual 16 GB Blackwell setup all tiers fit and run. The MEDIUM/HIGH/VERY-HIGH tiers additionally ship with higher-precision output tensors to reduce the rare repetition-loop failure mode described below. See the notes before treating any numbers as meaningful.
Vision works too. The GGUFs themselves are text-only, but the model is vision-capable: pair it with the vision projector from DavidAU's NEO-MAX-MTP-GGUF repo, unmodified. DavidAU publishes mmproj-BF16.gguf, mmproj-F16.gguf, and mmproj-F32.gguf; we used the BF16 one ourselves (verified byte-identical), but any of the three works. No separate mmproj upload is needed here; just point --mmproj at one of his files.
I post updates on new conversions, benchmarks, and what I'm working on over on my site. Follow along there to keep up with new releases and the work in progress. If you'd like to support more of it, you will find the options there. I do this on consumer hardware and like seeing how far it goes. More is on the way.
🌐 esatapedico.com. Updates, work-in-progress, and ways to support.
All tiers share the identical 496-tensor NVFP4 backbone (13.70 GB). They differ only in the 10 "extra" tensors (output.weight = LM head, token_embd.weight = embedding, and the 8 MTP draft-head tensors blk.64.*):
| Tier | File | lm_head (output.weight) | token_embd | MTP head | Size |
|---|---|---|---|---|---|
| LOW | ...-NVFP4-LOW.gguf | Q5_0 (0.87 GB) | IQ4_XS | IQ4_XS | 15.49 GB |
| MEDIUM | ...-NVFP4-MEDIUM.gguf | Q8_0 (1.35 GB) | Q6_K | IQ4_XS | 16.34 GB |
| HIGH | ...-NVFP4-HIGH.gguf | BF16 (2.54 GB) | Q6_K | IQ4_XS | 17.53 GB |
| VERY-HIGH | ...-NVFP4-VERY-HIGH.gguf | BF16 | BF16 | BF16 | 19.65 GB |
Tensor layout per tier (all 1,858 tensors; only the extra-tensor types differ):
| GGML type | Tensors | Size | Component |
|---|---|---|---|
NVFP4 | 496 | 13.70 GB | all 64 transformer blocks (attention QKV/output, FFN gate/up/down, gates); identical in all tiers |
F32 | 1,352 | 0.01 GB | norms, gates, scales |
| tier-dependent | 10 | see table | output.weight, token_embd.weight, blk.64.* MTP draft block (incl. nextn.eh_proj) |
The MTP draft head is embedded in the GGUF, so no separate drafter file is needed. Enable it in llama.cpp with --spec-type draft-mtp.
Why these extra tensors? The MTP draft head's job is to predict tokens the main model will accept. Keeping it at IQ4_XS instead of NVFP4 preserves draft quality, which is what makes speculative decoding pay off. The LM head precision is the main quality lever: Q5_0 (LOW) → Q8_0 (MEDIUM) → BF16 (HIGH/VERY-HIGH) progressively remove quantization noise from the exact tensor that determines the next-token distribution.
We previously recommended the DRY anti-loop sampler for the MEDIUM/HIGH/VERY-HIGH tiers; we no longer do — it interferes with verbatim reproduction of long strings (paths, identifiers, tool arguments), which matters for coding and tool use.
This is a derivative work built entirely from existing Apache-2.0 artifacts. Nothing here was trained or fine-tuned. Credit belongs to:
Q5_0/Q6_K output tensor and IQ4_XS extras) from DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF.The NVFP4 tensors are native GGML type 40, preserved from the source checkpoint through GGUF conversion with no re-quantization and no dequant-to-requant round trip.
convert_hf_to_gguf.py with --outtype auto (which preserves the native NVFP4 tensors as-is instead of dequantizing them). This produced the 19.65 GB all-NVFP4 backbone GGUF (the VERY-HIGH tier, unchanged).llama-quantize --tensor-type-file <overrides> from that parent, re-quantizing only the 10 extra tensors to the tier's chosen types and copying the NVFP4 backbone verbatim. The per-tier override maps are in this repo (overrides.txt, overrides-medium.txt, overrides-high.txt, overrides-very-high.txt).No weights were retrained or re-quantized beyond the extra tensors; the NVFP4 backbone is byte-identical in all four files.
Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-LOW.gguf (15.49 GB)Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-MEDIUM.gguf (16.34 GB)Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-HIGH.gguf (17.53 GB)Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-VERY-HIGH.gguf (19.65 GB)overrides.txt (LOW), overrides-medium.txt, overrides-high.txt, overrides-very-high.txt: the per-tensor quantization-type maps from our conversion planning (770 entries each: every weight tensor and its target GGML type, e.g. blk.0.attn_qkv.weight=nvfp4, norms pinned to f32). Useful if you want to reproduce or audit the tensor layout without re-deriving it.We did not run a proper benchmark. What follows are informal first impressions from a handful of single-stream runs, included only so others know what to expect. Do not treat these as claims.
split-mode: tensor, llama.cpp via LocalAI, flash attention on, quantized KV cache.--spec-type draft-mtp, spec_n_max 6, spec_p_min 0.75.The comparison rows:
--outtype auto conversion described above; now the VERY-HIGH tier), not a GGUF published by maci0.LOW tier (same file that shipped in the previous version of this repo).| maci0 NVFP4 (= VERY-HIGH) | IQ4XS LOW (DavidAU's GGUF) | NVFP4-LOW | |
|---|---|---|---|
| File size | 19.65 GB | 15.14 GB | 15.49 GB |
| Prefill (first impression) | 640.9 tok/s | 563.5 tok/s | 639.1 tok/s |
| Decode with MTP (first impression) | 15.98 tok/s | 19.78 tok/s | ~23.5 tok/s |
| MTP acceptance (first impression) | 0.880 | 0.878 | ~0.92 |
Each tier was measured with the same single 180k-token payload and a fresh LocalAI process (pod restart) so only one model was resident at a time. All tiers share the byte-identical NVFP4 backbone; only the 10 extra tensors differ.
| Tier | Prefill t/s | Decode t/s | MTP acceptance | Mean draft len |
|---|---|---|---|---|
| LOW | ~639 | ~19.2 | ~0.879 | ~3.0 |
| MEDIUM | ~642 | ~18.1 | ~0.878 | ~2.9 |
| HIGH | ~642 | ~16.8 | ~0.877 | ~3.1 |
| VERY-HIGH | ~643 | ~16.3 | ~0.867 | ~3.0 |
Naive first impressions, not a proper benchmark:
finish_reason: stop and
distinct-5-gram ratio >= 0.98 on the output. Repetition-analysis flags on the
repetitive-payload runs traced back to the model quoting source sentences verbatim,
not to degeneration.As with everything above: do not treat these as claims. Single-run, single-hardware, repetitive payload.
During development, one out of five long-generation runs degenerated: the model repeated itself until hitting max_tokens exactly. The signatures were:
finish_reason: length at exactly the token cap (20,000)Four subsequent runs, including the exact same 180k to 20k scenario on a fresh process each time, did not reproduce it, and all finished with finish_reason: stop. The cause is not yet fully understood; treat it as a rare, non-deterministic failure mode to watch for, not a guaranteed one. If you see these three signatures together in a long generation, this is likely what is happening.
Mitigation: see "Anti-loop sampling" above. The tier differences remain in the model-side precision of the extra tensors.
llama-server \
--model Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-MEDIUM.gguf \
--mmproj <optional: Qwen3.6-27B mmproj-BF16.gguf> \
--ctx-size 204800 \
--flash-attn on \
--spec-type draft-mtp \
--spec-draft-n-max 6 \
--spec-draft-p-min 0.75 \
--temp 0.6 --top-p 0.95 --top-k 20
sm_120 support (Blackwell).draft-mtp spec path (merged upstream as LLAMA_CONTEXT_TYPE_MTP).--mmproj at one of DavidAU's mmproj-BF16.gguf / mmproj-F16.gguf / mmproj-F32.gguf files from DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF (see the vision note above).--spec-draft-p-min 0.75 matters: without it the drafter wastes steps on low-confidence tokens and the speedup shrinks.--spec-draft-n-max values 1 through 6 and keep whatever is fastest on your system.q4_0/q4_0: ~3 GBApache-2.0, identical to every upstream artifact. The base model license governs; GGUF conversion and quantization are transformations, not new training. When redistributing, please retain attribution to Qwen (Alibaba), DavidAU, and maci0 as above.
"Qwen" is a trademark of Alibaba. Trademarks are used here only to identify upstream models; this repository is not affiliated with, sponsored by, or endorsed by Alibaba, DavidAU, or maci0.
This model card was written by an AI assistant at the request of the repository author, who did the engineering. As with any AI-generated text, there may be errors; please verify anything important (hashes, sizes, commands) against the file itself before relying on it.