Downloads · 30 days
59
0% of all-time downloads
kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16
Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16 is a image-text-to-text model from kkuspa. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This is the vLLM version of DavidAU's Qwen3.6-27B Fable-Fusion 711. The weights are FP4. Everything sensitive stays BF16. The repo is 28 GB instead of 56. It serves as an OpenAI-compatible endpoint on any NVIDIA GPU w…
Downloads · 30 days
59
0% of all-time downloads
All-time downloads
12.4K
Public
Parameters
27.4B
56.3 GB on disk
Likes
9
Public
Click a slice to open those files.
.safetensors28.6 GB · 100%
How the weights are stored.
U818.8B · 69%
From the Hugging Face model README
This is the vLLM version of DavidAU's Qwen3.6-27B Fable-Fusion 711. The weights are FP4. Everything sensitive stays BF16. The repo is 28 GB instead of 56. It serves as an OpenAI-compatible endpoint on any NVIDIA GPU with about 32 GB of VRAM, Ampere or newer. Blackwell is not required.
| Scheme | NVFP4A16 weight-only, compressed-tensors format, native vLLM support |
| Source | quantized directly from the BF16 safetensors. No GGUF step in the lineage. |
| Kept in BF16 | all 48 linear-attention (DeltaNet) layers, the vision tower, the MTP drafter, lm_head |
| Size | 27.7 GB weights (BF16 parent: 55.6 GB) |
| VRAM | ~32 GB minimum; tested on 1x RTX PRO 6000 Blackwell (96 GB) |
| Context | up to 262,144 tokens, VRAM permitting |
| Vision | included, unquantized |
| MTP speculative decoding | verified: 1.56x decode speedup at draft depth 5 |
| FP8 KV cache | calibrated k_scale/v_scale included; verified with --kv-cache-dtype fp8 |
| License | apache-2.0, inherited from the parent |
vllm serve kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16 \
--max-model-len 32768 --gpu-memory-utilization 0.85
Tested on vLLM 0.26.0. The server selects the Marlin NVFP4 kernel and warns that the GPU lacks native FP4 compute. The warning is expected: activations are BF16 by design, so the FP4 tensor-core path does not apply.
For faster decoding, enable the built-in MTP drafter:
vllm serve kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16 --max-model-len 32768 --gpu-memory-utilization 0.85 --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":5}'
DavidAU tuned this model for specific sampler settings. They translate to the OpenAI API as follows:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
# Thinking mode (default template): creative work, reasoning
r = client.chat.completions.create(
model="kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16",
messages=[{"role": "user", "content": "Open a noir story set in a lighthouse."}],
temperature=1.0, top_p=0.95, max_tokens=2048,
extra_body={"top_k": 20, "min_p": 0.0, "repetition_penalty": 1.0},
)
# Coding: temperature=0.6, same top_p and top_k
# Instruct mode: temperature=0.7, top_p=0.80, presence_penalty=1.5
The repo ships two chat templates. chat_template.jinja (thinking) is the default. For instruct-style responses without a thinking block, pass --chat-template pointing at chat_template-instruct.jinja at serve time.
At the time of publication this is the only quant of Fable-Fusion 711 that stock vLLM can serve. The other safetensors quants of this model use a custom runtime format.
Measured with lm-evaluation-harness 0.4.12 (vLLM backend, 0-shot, batch auto) against the BF16 parent on the same hardware:
| Task | BF16 | NVFP4A16 | Delta |
|---|---|---|---|
| ARC-Challenge (acc_norm) | 0.6126 | 0.6203 | +0.008 |
| HellaSwag (acc_norm) | 0.8502 | 0.8490 | -0.001 |
| Winogrande (acc) | 0.7806 | 0.7814 | +0.001 |
Deltas sit inside the standard error on every task (ARC-Challenge stderr is 0.014). The quantization is a data-free RTN pass. No calibration dataset was used, so nothing was tuned toward any benchmark. Absolute scores depend on harness settings; the point of this table is the like-for-like comparison, both configs measured 0-shot on the same harness and hardware.
Single request, 512-token greedy generations, 1x RTX PRO 6000 Blackwell (96 GB), vLLM 0.26.0:
| Config | Decode speed |
|---|---|
| Standard decoding | 56.2 tok/s |
| MTP speculative, depth 5 | 87.4 tok/s (1.56x) |
Acceptance across the test: 996 of 3000 drafted tokens (about 1.7 extra tokens per draft window). Throughput under concurrent load will differ.
llm-compressor 0.12.0, one-shot:
QuantizationModifier(
targets="Linear",
scheme="NVFP4A16",
ignore=["lm_head", "re:.*visual.*", "re:.*linear_attn.*", "re:.*mtp.*"],
)
Only the Linear weights in the full-attention and MLP blocks carry FP4. The DeltaNet linear-attention layers, the vision tower, the MTP drafter, and lm_head keep BF16, because hybrid-attention layers and drafters degrade badly under 4-bit quantization. The exact recipe ships in recipe.yaml.
preprocessor_config.json and video_preprocessor_config.json here come from Qwen/Qwen3.6-27B.model-mtp-bf16.safetensors and are wired into model.safetensors.index.json. Transformers drops them on load; they were re-extracted from the parent checkpoint so speculative decoding remains possible.The parent model is refusal-ablated ("Uncensored", "Heretic") and tuned for unconstrained creative writing. This quant changes none of that. It will produce content that aligned models refuse. You are responsible for how you deploy it.
Qwen/Qwen3.6-27B by the Qwen team, then the Fable-Fusion 711 multi-stage tune and merge by DavidAU with collaborators credited on the parent card, then this quantization. Thanks to all of them. Apache-2.0 throughout.