Downloads · 30 days
18
8% of all-time downloads
j-a-a-a-y/Huihui-Qwen3.5-27B-abliterated-CT-W4A16
Huihui-Qwen3.5-27B-abliterated-CT-W4A16 is a machine learning model from j-a-a-a-y. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
4-bit weight-only quantization of huihui-ai/Huihui-Qwen3.5-27B-abliterated.
Downloads · 30 days
18
8% of all-time downloads
All-time downloads
238
Public
Parameters
27.5B
18.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors18.5 GB · 100%
How the weights are stored.
I3224.4B · 89%
From the Hugging Face model README
4-bit weight-only quantization of huihui-ai/Huihui-Qwen3.5-27B-abliterated.
Text-only: vision encoder removed to save 0.9 GB VRAM.
MTP preserved: speculative decoding works out of the box.
| Property | Value |
|---|---|
| Base model | Qwen3.5-27B (abliterated) |
| Quantization | W4A16 (compressed-tensors, group_size=128) |
| Size on disk | 18.5 GB |
| vs GPTQ W4A16 | -1.0 GB (no vision encoder) |
| MTP head | Included (BF16, 0.85 GB) |
| Vision encoder | Removed |
| Calibration | 128 samples, Pile validation |
python -m vllm.entrypoints.openai.api_server \
--model j-a-a-a-y/Huihui-Qwen3.5-27B-abliterated-CT-W4A16 \
--served-model-name qwen3.5-27b \
--dtype float16 \
--max-model-len 1024 \
--max-num-seqs 4 \
--gpu-memory-utilization 0.97 \
--speculative-config '{"method": "mtp", "num_speculative_tokens": 5}' \
--performance-mode interactivity
| Metric | GPTQ W4A16 | This model |
|---|---|---|
| Single 256 tok | 149 tok/s | TBD |
| Single 512 tok | 131 tok/s | TBD |
| Batch=4 agg | 410 tok/s | TBD |
| MTP acceptance | 50% | TBD |
| Available KV cache | 6.35 GiB | ~7.3 GiB (est.) |