Qwen3.8-27B-Dominatrix-NVFP4A16

Qwen 3.8 27B Dominatrix (NVFP4A16)

NVFP4A16 quantization of allura-org/Qwen3.8-27B-Dominatrix (a roleplay finetune of Qwen/Qwen3.8-27B by Fizz / Allura).

Produced with llm-compressor 0.13.0, data-free PTQ, scheme NVFP4A16 (W4A16, group size 16), exported as compressed-tensors (nvfp4-pack-quantized).

Intended for vLLM on NVIDIA Blackwell / DGX Spark (GB10).

What is quantized

• Targets: Linear layers (Qwen3_5ForConditionalGeneration)
• Quantized: full-attention projections and MLPs
• Left high-precision: lm_head, vision tower, GDN linear_attn internals, norms, embeddings

~51 GB BF16 → ~26 GB NVFP4A16.

Serve with vLLM

vllm serve logicauro/Qwen3.8-27B-Dominatrix-NVFP4A16 \
  --served-model-name Qwen3.8-27B-Dominatrix \
  --language-model-only \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8 \
  --max-model-len 98304 \
  --max-num-seqs 1 \
  --reasoning-parser qwen3 \
  --chat-template chat_template.jinja \
  --trust-remote-code

Sampling (from the original card)

Temperature 1.0–1.25 with min_p 0.1 or top_p 0.95. Chat-template defaults: preserve_thinking off, reasoning_effort medium.

Credits

Original finetune: allura-org/Qwen3.8-27B-Dominatrix (Fizz / Allura), base Qwen/Qwen3.8-27B.