Downloads · 30 days
76
21% of all-time downloads
zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated-MTP-NVFP4
Qwen3.8-27B-Dominatrix-abliterated-MTP-NVFP4 is a image-text-to-text model from zebulon-prime. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Mixed-precision NVFP4/FP8 PTQ of zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated — allura-org's Dominatrix roleplay finetune with huihui-ai's refusal direction projected out.
Downloads · 30 days
76
21% of all-time downloads
All-time downloads
360
Public
Parameters
18.8B
23.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors23.7 GB · 100%
How the weights are stored.
U88.6B · 45%
From the Hugging Face model README
Mixed-precision NVFP4/FP8 PTQ of zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated — allura-org's Dominatrix roleplay finetune with huihui-ai's refusal direction projected out.
23 GB, built with NVIDIA TensorRT Model Optimizer for Blackwell (SM120) inference in SGLang or vLLM.
The distinguishing feature: lm_head is left dense BF16, which is a hard requirement for
DFlash 2 speculative decoding. Most ModelOpt NVFP4 exports of this architecture quantize
lm_head and therefore cannot run DFlash 2 at all. Cost of the dense head is ~1.9 GB of
VRAM over a packed one.
| component | precision |
|---|---|
MLP gate_proj / up_proj / down_proj | NVFP4 W4A4 |
self_attn q/k/v/o, linear_attn projections | FP8 e4m3 |
| KV cache | FP8 |
lm_head | BF16, dense |
embed_tokens, MTP head, vision tower | BF16 |
Export format is ModelOpt MIXED_PRECISION with a per-layer map in hf_quant_config.json.
Calibrated on in-domain ChatML roleplay text rather than a generic news corpus.
hf_quant_config.jsonrecordsproducer.version: 0.0.0because it was built from an editable install. That field is not meaningful provenance.
Requires the z-lab/Qwen3.8-27B-DFlash2 drafter and an SGLang build including PR #35371.
sglang serve \
--trust-remote-code \
--model-path /models/Qwen3.8-27B-Dominatrix-abliterated-MTP-NVFP4 \
--mem-fraction-static 0.70 \
--attention-backend flashinfer \
--chunked-prefill-size 2048 \
--reasoning-parser qwen3 \
--speculative-algorithm DFLASH \
--speculative-draft-model-path /models/Qwen3.8-27B-DFlash2-zlab \
--speculative-dflash-block-size 8 \
--speculative-draft-model-quantization unquant
--kv-cache-dtype can be omitted: this checkpoint declares kv_cache_quant_algo: FP8, so
SGLang's auto resolves it.
Drop the four --speculative-* flags. The dense lm_head buys nothing in that configuration
but is otherwise harmless.
The mtp.* tensors survive the quant in BF16, so MTP speculation remains available as an
alternative drafter. Pick one — MTP or DFlash 2, not both.
NVFP4 requires Blackwell (SM120+) for native FP4 tensor-core execution. Built and tested on an RTX PRO 6000 Blackwell. Weights are ~23 GB, leaving room for a long-context KV cache and the 2B DFlash 2 drafter.
Fidelity of the underlying BF16 abliteration versus stock Dominatrix is summarised on the BF16 card.
The quantization error of this build has not been measured. Treat it as unquantified.
Leaving lm_head dense should help, since the output projection is among the most
quantization-sensitive layers, but that is reasoning, not a measurement.
Sampler guidance from upstream Dominatrix carries over: temperature 1.0–1.25 with min_p 0.1 or top_p 0.95; some prefer 0.7 and nothing else.