Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4

Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4

NVFP4 vLLM-first build of HauhauCS/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive, with MTP tensors from the official Qwen/Qwen3.6-27B checkpoint and the full visual tower preserved in floating point.

Lineage

Contents

• NVFP4 text backbone shards for vLLM compressed-tensors loading.
• model-multimodal-extra.safetensors containing:
  • 333 model.visual.* tensors for the visual tower.
  • 15 mtp.* tensors for speculative decoding experiments.
• Runtime-compatible model.safetensors.index.json with no stray model.language_model.visual.* entries.

Recommended vLLM Startup

Full multimodal profile:

VLLM_NVFP4_GEMM_BACKEND=marlin \
vllm serve /path/to/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4 \
  --host 0.0.0.0 \
  --trust-remote-code \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.92 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --kv-cache-dtype fp8

Text-only long-context profile:

VLLM_NVFP4_GEMM_BACKEND=marlin \
vllm serve /path/to/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4 \
  --host 0.0.0.0 \
  --trust-remote-code \
  --language-model-only \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.92 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --kv-cache-dtype fp8

MTP speculative profile:

VLLM_NVFP4_GEMM_BACKEND=marlin \
vllm serve /path/to/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4 \
  --trust-remote-code \
  --language-model-only \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.92 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

Agent-Safe Responses Template

For agent loops that use /v1/responses, the stock Qwen3.6 template may enter thinking mode by default and spend the whole output budget on reasoning. This repo includes an optional deployment template:

chat_template_responses_reasoning_switch.jinja

Recommended agent-safe startup:

VLLM_NVFP4_GEMM_BACKEND=marlin vllm serve /path/to/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-NVFP4   --host 0.0.0.0   --trust-remote-code   --language-model-only   --chat-template chat_template_responses_reasoning_switch.jinja   --max-model-len 131072   --gpu-memory-utilization 0.92   --max-num-seqs 1   --max-num-batched-tokens 4096   --kv-cache-dtype fp8

Behavior with this template:

• /v1/responses without reasoning: non-thinking by default; avoids hidden-reasoning leakage in agent loops.
• /v1/responses with reasoning: {"effort": "low|medium|high"}: opens Qwen thinking mode via the template variable reasoning_effort.
• This does not patch vLLM. Requested thinking is emitted as normal output text unless you run a separate parser/wrapper path.

Validation Status

• Tested on RTX 5090 with vllm/vllm-openai:cu130-nightly.
• VLLM_NVFP4_GEMM_BACKEND=marlin is validated for this fixed artifact across text-only, full multimodal, and MTP startup paths.
• flashinfer-cutlass also works and may be preferable for throughput tuning on this RTX 5090.
• Full multimodal startup succeeds with Qwen3_5ForConditionalGeneration.
• Visual loading no longer emits Parameter visual.blocks... not found in params_dict.
• /v1/responses image input was tested successfully against a 1980x1160 PNG screenshot.
• MTP tensors are included and startup has been validated with both marlin and flashinfer-cutlass; acceptance and throughput depend on workload and vLLM tuning.

Notes

• The visual tower is intentionally not NVFP4-quantized; it is preserved separately in the multimodal extra shard.
• Qwen3.6 linear-attention layers such as linear_attn.in_proj_a / linear_attn.in_proj_b remain BF16, so their non-64-aligned output dimensions do not enter the NVFP4 Marlin GEMM path.
• For agentic long-context text serving on a single 5090, use --language-model-only to avoid reserving vision-side memory.
• Current vLLM Responses behavior may still expose Qwen thinking text depending on chat-template handling; applications should parse and post-process accordingly.

References

• Source finetune GGUF: https://huggingface.co/HauhauCS/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive
• Official base model: https://huggingface.co/Qwen/Qwen3.6-27B
• Conversion and quantization pipeline: https://github.com/li-yifei/gguf-to-nvfp4
• vLLM deployment config for RTX 5090: https://github.com/li-yifei/vllm-qwen3.6-nvfp4-5090