Downloads · 30 days
77
14% of all-time downloads
tacodevs/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-FP8
Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-FP8 is a image-text-to-text model from tacodevs. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Calibrated FP8 quantization of DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP that preserves vision-language capabilities, unlike vLLM's dynamic FP8 which destroys them on this architecture family.
Downloads · 30 days
77
14% of all-time downloads
All-time downloads
559
Public
Parameters
27.4B
30.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors30.4 GB · 100%
How the weights are stored.
F8_E4M324.4B · 89%
From the Hugging Face model README
Calibrated FP8 quantization of DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP that preserves vision-language capabilities, unlike vLLM's dynamic FP8 which destroys them on this architecture family.
--quantization fp8 to vLLM — quantization config is baked in via compressed-tensors.chat_template_kwargs: {"enable_thinking": false} — see Usage. Without it the model emits long tag-less "Thinking Process:" reasoning before answering.The source is DavidAU's multi-stage merge on the Qwen3.6-27B base (Qwen qwen3_5 architecture class: hybrid attention with Gated DeltaNet linear-attention layers + full attention every 4th layer, vision encoder included, 256k native context). Decensoring is via Heretic v1.2 with Arbitrary-Rank Ablation (ARA) — the source card measures 4/100 refusals vs 99/100 for the base model.
Note: the source repo ships a sidecar MTP (multi-token prediction) tensor file that is not part of the model index. It is not included in this quant; vLLM does not use it for standard serving.
vLLM's runtime --quantization fp8 uses a single tensor-wide scale per Linear layer. The vision merger — the only bridge between the 1152-dim visual tower and the 5120-dim LM embedding space — has a much wider weight distribution than LM layers, so single-scale FP8 rounds its small-magnitude weights to zero. The LM then receives noise at image-token positions and silently hallucinates image descriptions from text alone. See the detailed write-up in the sibling repo tacodevs/Huihui-Qwen3.5-27B-Claude-4.6-Opus-abliterated-FP8.
linear_attn modules kept in BF16 (hybrid-attention internals are not plain GEMMs and are excluded).Only the language model body (full-attention projections and MLP linears) is FP8.
recipe.yaml in this repo)FP8_DYNAMIC (per-channel weight scales, dynamic per-token activation scales)Linear layersre:.*visual.* (visual tower + merger), all linear_attn modules and norms, lm_headpython -m vllm.entrypoints.openai.api_server \
--model tacodevs/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-FP8 \
--max-model-len 16384 \
--gpu-memory-utilization 0.90 \
--trust-remote-code
IMPORTANT: Do NOT pass --quantization fp8. The model already has its quantization config baked in via compressed-tensors; vLLM detects and uses the proper FP8 path automatically. Passing --quantization fp8 would re-quantize the already-FP8 weights and break everything.
The chat template supports Qwen's enable_thinking switch. By default the model produces extended reasoning without <think> tags (plain "Thinking Process:" markdown), which is easy to overrun token budgets with and hard to strip in streaming pipelines. For structured-output or latency-sensitive tasks, disable it per request:
{
"model": "tacodevs/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-FP8",
"chat_template_kwargs": {"enable_thinking": false},
"messages": [...]
}
In our structured image-prompt workload this cut output from a truncated 500+ tokens to a complete 130–210 tokens and total latency from ~11 s to ~4.7 s, with format-perfect results.