Downloads · 30 days
34
6% of all-time downloads
will1208/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 is a image-text-to-text model from will1208. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
NVFP4 quantized version of HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive.
Downloads · 30 days
34
6% of all-time downloads
All-time downloads
532
Public
Parameters
20.8B
46.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors23.4 GB · 100%
How the weights are stored.
U816.3B · 78%
From the Hugging Face model README
NVFP4 quantized version of HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive.
Conservative profile: linear_attn (30 DeltaNet/Mamba layers) and MTP kept in bf16 for best quality. Follows AEON-7/RedHatAI approach.
| Spec | Value |
|---|---|
| Base model | HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive (Q8_K_P GGUF) |
| Original model | Qwen/Qwen3.6-35B-A3B |
| Architecture | Qwen3.5 MoE — 35B total, 3B active, 256 experts (8 routed + 1 shared) |
| Quantization | NVFP4 W4A4 (conservative: linear_attn + MTP in bf16) |
| Format | compressed-tensors (native vLLM support) |
| Size | ~22 GB |
| Max context (text-only) | 131K+ on RTX 5090 |
| Requires | NVIDIA Blackwell GPU (SM 120) |
recipe = QuantizationModifier(
targets="Linear", scheme="NVFP4",
ignore=["lm_head", "re:.*visual.*", "re:.*mlp.gate$",
"re:.*mlp.shared_expert_gate$", "re:.*linear_attn.*", "re:^mtp.*"],
)
oneshot(model=model, dataset=ds, recipe=recipe,
max_seq_length=1024, num_calibration_samples=128,
moe_calibrate_all_experts=True, pipeline="basic")
This repository has been smoke-tested locally on an RTX 5090 with vllm/vllm-openai:v0.21.0-cu130-local, compressed-tensors, NVFP4 Marlin GEMM, FP8 KV cache, and a real image chat.completions request.
VLLM_USE_FLASHINFER_MOE_FP4=0 \
VLLM_NVFP4_GEMM_BACKEND=marlin \
vllm serve ./Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \
--served-model-name qwen36-35b-a3b-hauhaucs-nvfp4 \
--quantization compressed-tensors \
--kv-cache-dtype fp8 \
--gpu-memory-utilization 0.90 \
--max-model-len 4096 \
--max-num-seqs 1 \
--max-num-batched-tokens 1024 \
--trust-remote-code
For short non-thinking answers, pass chat_template_kwargs at the top level of the OpenAI-compatible request:
{
"chat_template_kwargs": {"enable_thinking": false}
}
VLLM_USE_FLASHINFER_MOE_FP4=0 \
VLLM_NVFP4_GEMM_BACKEND=marlin \
vllm serve ./Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \
--quantization compressed-tensors \
--kv-cache-dtype fp8 \
--gpu-memory-utilization 0.95 \
--max-model-len 100000 \
--max-num-seqs 1 \
--reasoning-parser qwen3 \
--language-model-only \
--trust-remote-code
Converted using li-yifei/gguf-to-nvfp4:
Q8_K_P GGUF → step1_convert_qwen36_moe.py → HF bf16 → step2_quantize_qwen36_moe.py → NVFP4