Downloads · 30 days
1.5K
4% of all-time downloads
THe-Plague/Qwen3.6-35B-A3B-abliterated-NVFP4-MTP
Qwen3.6-35B-A3B-abliterated-NVFP4-MTP is a text generation model from THe-Plague. Use it when you need the model to write or continue text. It is set up for vllm. The card lists the license as apache-2.0.
The fastest abliterated Qwen3.6 for NVIDIA Blackwell / DGX Spark — NVFP4 quantized with MTP (Multi-Token Prediction) layers for speculative decoding.
Downloads · 30 days
1.5K
4% of all-time downloads
All-time downloads
36.3K
Public
Parameters
20.4B
23.6 GB on disk
Likes
10
Public
Click a slice to open those files.
.safetensors23.6 GB · 100%
How the weights are stored.
U816.8B · 82%
From the Hugging Face model README
The fastest abliterated Qwen3.6 for NVIDIA Blackwell / DGX Spark — NVFP4 quantized with MTP (Multi-Token Prediction) layers for speculative decoding.
95.2 tok/s decode on a single DGX Spark (GB10) with MTP=3.
No other model on HuggingFace combines all three:
| Feature | This Model | sakamakismile NVFP4 | RedHatAI NVFP4 |
|---|---|---|---|
| NVFP4 weights | ✅ | ✅ | ✅ |
| MTP layers | ✅ BF16 | ❌ | ✅ |
| Abliterated | ✅ | ✅ | ❌ |
| Speculative decode | ✅ 95 tok/s | ❌ ~70 tok/s | ✅ but censored |
Without MTP layers, you're limited to ~70 tok/s on GB10. MTP enables speculative decoding which gives a +35% speedup.
model-mtp.safetensors)model-mtp.safetensors filemodel.safetensors.index.json with MTP weight mappingsHardware: NVIDIA DGX Spark (GB10 Blackwell, sm_121, ~128GB unified memory) Engine: vLLM v0.23.1 with FlashInfer + CUTLASS NVFP4 MoE kernels Benchmark: 256 generated tokens at various context lengths
| Config | 1K ctx | 4K ctx | 16K ctx | 32K ctx | 64K ctx |
|---|---|---|---|---|---|
| MTP=3, seqs=1 (best) | 92.2 | 95.2 | 90.7 | 89.7 | 84.4 |
| MTP=2, seqs=4 | 87.2 | 75.5 | 82.8 | 81.6 | 74.4 |
| MTP=4, seqs=1 | 75.6 | 71.3 | 74.0 | 77.6 | 74.4 |
| No MTP (NVFP4 only) | ~71 | ~70 | ~74 | ~68 | ~70 |
MTP=3 is the sweet spot. MTP=4 regresses due to speculation overhead exceeding acceptance rate.
docker run -d --gpus all --network host --ipc host --shm-size=16g \
--name vllm-qwen36 \
-e VLLM_MARLIN_USE_ATOMIC_ADD=1 \
-e TORCH_MATMUL_PRECISION=high \
-v /path/to/models:/models \
ghcr.io/spark-arena/dgx-vllm-eugr-nightly:latest \
vllm serve /models/Qwen3.6-35B-A3B-abliterated-NVFP4-MTP \
--host 0.0.0.0 --port 8000 --max-model-len 131072 \
--max-num-batched-tokens 32768 --max-num-seqs 1 \
--trust-remote-code --gpu-memory-utilization 0.7 \
--reasoning-parser qwen3 --kv-cache-dtype fp8 \
--load-format instanttensor --attention-backend flashinfer \
--speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' \
--enable-prefix-caching -tp 1