Downloads · 30 days
98
13% of all-time downloads
blockblockblock/Huihui-Qwen3.6-35B-A3B-abliterated-exl3-4.0bpw
Huihui-Qwen3.6-35B-A3B-abliterated-exl3-4.0bpw is a text generation model from blockblockblock. Use it when you need the model to write or continue text. It is set up for exllamav3. The card lists the license as apache-2.0.
Downloads · 30 days
98
13% of all-time downloads
All-time downloads
749
Public
Parameters
9.7B
19.5 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors19.5 GB · 100%
How the weights are stored.
I168.7B · 89%
From the Hugging Face model README
<sub><code>EXL3</code> · <b>4.0 bpw</b> · 19.5 GB · Mixture‑of‑Experts · 40 layers × 256 experts</sub>
<br/> </div>[!NOTE] An ExLlamaV3 build of
huihui-ai/Huihui-Qwen3.6-35B-A3B-abliteratedat 4.0 bits per weight — balanced build — comfortable on a 24 GB consumer GPU (~20 GB weights). See Quants for sibling repos at other bit‑widths or browse the collection.
| BPW | Head bits | Calibration rows | Size | Status |
|---|---|---|---|---|
| 4.0 | 8 | 128 | 19.5 GB | <kbd>this repo</kbd> |
| 4.5 | 8 | 250 | 21.6 GB | link |
[!TIP] VRAM at 4.0 bpw: ~19.5 GB weights + ~2 GB context. Comfortable on a single 24 GB card; tight on 16 GB with reduced context.
pip install -U huggingface_hub
hf download \
blockblockblock/Huihui-Qwen3.6-35B-A3B-abliterated-exl3-4.0bpw \
--local-dir ./Huihui-Qwen3.6-35B-A3B-abliterated-exl3-4.0bpw
<details>
<summary><b>Quantization recipe</b> <sub>(advanced — embedded in <code>quantization_config.json</code>)</sub></summary>
<br/>
| Setting | Value |
|---|---|
| Format | EXL3 |
| Bits per weight | 4.0 |
| Head bits | 8 |
| Calibration rows | 128 |
| Codebook | MCG |
| Out‑scales | always |
| Parallel mode | enabled (MoE expert batching) |
Loaded automatically by every ExLlamaV3 loader; reproduced here for searchability.
</details>[!IMPORTANT] Use and license follow the base model. Quantization adds no additional restrictions. Refer to the upstream repository for terms, citation, and safety documentation.