Qwen3.8-27B-TURBO-Fable-MXFP4-awq

Qwen3.8-27B-TURBO-Fable-MXFP4-awq

Quark MXFP4 (W4A8) + AWQ quantization of DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU — a 27B dense model built on Qwen/Qwen3.8-27B via multi-stage "Cold Fusion" / GAIN training with an uncensored (abliterated-style) heretic treatment.

Same recipe as just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq — this is that pipeline applied to the Fable Cold-Fusion source.

• Full BF16 source: ~55 GB → this checkpoint: 18.04 GiB (single model.safetensors)
• Verified serving locally on vLLM (AMD RDNA4 / gfx1201, TP2) with coherent output
• Quantized with AMD Quark 0.12 (quant_scheme mxfp4, quant_algo awq, 64 calibration samples, seq 512, pileval)

How to run

Start with this image: https://hub.docker.com/r/stilldeadcode/vllm-radiance Add this: https://codeberg.org/ggz14/radiance-vllm-mxfp4 NOTE: fp8_mtp.py was already ran on this model, it is not needed to run again.

Quantization details

What the source model is

Per the source card: a multi-stage tuned model combining Cold Fusion 735/882 ratio training, GAIN training, and an uncensored "NM/DAU" heretic treatment on the Qwen3.8-27B base — aimed at instruction following, creative prose, and unrestricted assistant behavior. Refer to the source model card for full details on the training method and intended use. This repo only changes the precision (BF16 → MXFP4/AWQ); weights are otherwise derived from that source.

Launching with vLLM

Standard OpenAI-compatible server:

vllm serve /path/to/Qwen3.8-27B-TURBO-Fable-MXFP4-awq \
  --host 0.0.0.0 --port 8000 \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90 \
  --trust-remote-code

Notes:

• Serving stack matters more than usual. Quark MXFP4 W4A8 needs a vLLM build with the Quark/MXFP4 path enabled. On AMD RDNA4 this was validated on a radiance vLLM 0.27.1 (torch 2.11+rocm7.14) build with the MXFP4 custom kernels; stock upstream vLLM with quant_method: quark support should load it via its Quark integration — if your build lacks MXFP4 kernels for your GPU, you will get a kernel-availability error at load, not silent slowdown.
• The checkpoint carries a full Qwen3.8 chat template (chat_template.jinja included) with reasoning-effort control (xhigh/medium/low, default xhigh) and XML tool calling. Pass chat_template_kwargs: {"enable_thinking": false} to disable thinking mode.
• 32 GB-class GPUs: fits in TP2 on 2×32 GB (18 GiB weights + activations + FP8 KV cache). Single 32 GB card is too tight for comfortable context; single 48 GB+ works.
• Speculative decoding: the FP8 MTP head is already wired in the config; --speculative-config '{"method":"mtp","num_speculative_tokens":5}' works on stacks that support Qwen MTP drafting.

Verification performed

• Quant output gates: quant_method: quark, fp4/group-32 weights + activations, exclude = lm_head + vision, self_attn U8-packed, 18.04 GiB (matches AMD's reference size for this architecture — no accidental exclusions)
• Converted layout verified: 1703 tensors, correct multimodal wrapper config, algo_config nulled (vLLM quark mapper compatibility), FP8 MTP declared
• Served end-to-end on vLLM (RDNA4 TP2): healthy boot, coherent greedy completions in both thinking and non-thinking modes

Intended use & limitations

Uncensored model — intended for adults, research, and responsible use; no built-in safety alignment. All limitations of the source apply (see its card). Quantization (MXFP4 W4A8) adds minor degradation vs BF16; AWQ scaling was calibrated to minimize output deviation on general text, but has not been benchmarked against the BF16 source with formal metrics.

Credits

• Source model: DavidAU — Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
• Base model: Qwen — Qwen3.8-27B
• Quantization: AMD Quark (MXFP4 + AWQ), AWQ scaling map from AMD's Qwen3.8-27B-Quark-AWQ-MXFP4 reference checkpoint