Qwen3.8-27B-NVFP4-Uncensored

Qwen3.8-27B-NVFP4-Uncensored

This is a ModelOpt mixed-precision derivative of orcarouter/Qwen3.8-27B-Uncensored, generated from source revision 9878936be9458522b5aeed0e13476bb8426f57f0.

This is not a pure all-NVFP4 checkpoint. Its weight path combines NVFP4, FP8, and retained higher-precision tensors; the exact composition is described below. The repository is intended to be a reproducible vLLM/ModelOpt artifact, not a claim that every Transformers backend can execute the quantized layers without matching ModelOpt support.

The model preserves the Qwen3.8 vision-language tower and MTP head. It is an abliterated/uncensored model with substantially reduced built-in refusal behavior. It is intended for controlled research, evaluation, and local experimentation. Add your own moderation and access controls before any user-facing deployment.

Quantization

• MLP and language-model-head weights: W4A16_NVFP4, group size 16. The exported metadata identifies 193 such target layers.
• Attention and linear-attention projections: ModelOpt FP8 mixed precision, covering 208 target layers.
• KV cache: FP8 E4M3 with 32 exported scalar scale tensors (16 K and 16 V) from the data-calibrated ModelOpt kv_fp8 recipe. The audited scales are finite, positive, and non-unit.
• Vision, MTP, and hybrid-state tensor families are retained in the unified checkpoint and excluded from the weight-quantization target map.
• Export format: unified Hugging Face safetensors checkpoint.

The full export contains 2,033 indexed tensors across three safetensors shards. ModelOpt metadata records MIXED_PRECISION weights and kv_cache_quant_algo: FP8, produced with ModelOpt 0.47.0.dev81+ga2fbac7ba.

Calibration

The KV scales were calibrated with 256 pre-rendered, text-only examples at a 2,048-token calibration sequence length, batch size 1, and enable_thinking=false. The corpus was generic rather than application-owned: 128 general instruction/chat rows, 64 code/technical rows, 32 structured-output rows, and 32 longer-context rows. No images or videos were used for calibration. This establishes a scale-aware FP8-KV export for the tested distribution; it is not a universal multimodal or application-specific calibration claim.

vLLM serving

This artifact was validated with vllm/vllm-openai:v0.27.1. The relevant starting flags are:

--quantization modelopt_fp4
--kv-cache-dtype fp8_e4m3
--max-model-len 262144
--trust-remote-code

On the validation stack, the requested modelopt_fp4 flag resolved to vLLM's modelopt_mixed path. The resolved KV dtype was float8_e4m3fn.

For the tested Qwen XML tool-calling route, also use:

--enable-auto-tool-choice --tool-call-parser qwen3_xml
--default-chat-template-kwargs '{"enable_thinking":false}'

Validation notes

In the project's Experiment 014, the candidate returned 56/56 HTTP 200 responses with complete streaming [DONE] markers, zero stream parse errors, and zero reasoning leaks. The explicit response-contract path passed 8/8. The unchanged no-contract baseline passed 12/24 because the known fenced-JSON and HH:MM formatting behaviors remained; these were output contract misses, not cache-load failures. A short long-context load returned 24/24 complete streams at concurrency levels 1, 2, and 4; its diagnostic fixture oracle passed 21/24. vLLM reported an allocator capacity of 2,491,134 FP8-KV tokens while the configured maximum sequence length was 262,144. The allocator figure is a host/runtime capacity diagnostic, not a claim that the model supports a 2.49-million-token context.

These are narrow, single-host exploratory results, not a general quality or production-readiness claim.

Important caveats

• On the project's NVIDIA GB10, vLLM uses the Marlin software-FP4 path because the GPU has no native FP4 computation support; compute-heavy performance may differ from a native-FP4 GPU.
• The checkpoint does not include separate q-scale metadata. In the tested vLLM FP8 attention path, q scaling therefore falls back to the K scale; q/probability scales were not independently calibrated.
• The calibration and validation were text-focused. Multimodal quality, application-wide quality, independent BF16-vs-FP8 throughput, and production readiness remain unproven. The tested persistent WebUI route continued to use BF16 KV as its correctness-first default.
• The model's uncensored behavior means it may produce harmful or illegal content. Follow the Apache 2.0 license, applicable law, and your own safety requirements.

License and provenance

The Apache 2.0 license file from the derived checkpoint is included in this repository. Users are responsible for complying with the base model's terms, the derivative model's terms, and all applicable laws.