Qwen3.6-35B-A3B-exl3-5.0bpw

blockblockblock/Qwen3.6-35B-A3B-exl3-5.0bpw

Qwen3.6-35B-A3B-exl3-5.0bpw is a text generation model from blockblockblock. Use it when you need the model to write or continue text. It is set up for exllamav3. The card lists the license as apache-2.0.

Downloads · 30 days

4% of all-time downloads

All-time downloads

548

Public

Parameters

11.8B

23.7 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors23.7 GB · 100%

Parameter types

How the weights are stored.

I1610.8B · 91%

Try a prompt

Base models

Qwen/Qwen3.6-35B-A3B

Model card

From the Hugging Face model README

Qwen3.6 · 35B-A3B

<sub><code>EXL3</code> · <b>5.0 bpw</b> · 23.7 GB · Mixture‑of‑Experts · 40 layers × 256 experts</sub>

<br/>

</div>

[!NOTE] An ExLlamaV3 build of Qwen/Qwen3.6-35B-A3B at 5.0 bits per weight — the quality build — fits a 24 GB card with reduced context, ideal headroom on 32 GB cards. See Quants for sibling repos at other bit‑widths or browse the collection.

Quants

BPW	Head bits	Calibration rows	Size	Status
5.0	8	250	23.7 GB	<kbd>this repo</kbd>

</div>

Inference

<table> <thead> <tr> <th align="left" width="32%">Loader</th> <th align="left">Use it for</th> </tr> </thead> <tbody> <tr> <td><a href="https://github.com/theroyallab/tabbyAPI"><b>TabbyAPI</b></a></td> <td>OpenAI‑compatible HTTP server. Drop‑in for OpenAI clients.</td> </tr> <tr> <td><a href="https://github.com/oobabooga/text-generation-webui"><b>text‑generation‑webui</b></a></td> <td>Local chat UI. Pick the <i>ExLlamaV3</i> loader from the model dropdown.</td> </tr> <tr> <td><a href="https://github.com/turboderp-org/exllamav3"><b>ExLlamaV3</b></a></td> <td>Direct Python API for embedding the model in your own code or pipeline.</td> </tr> </tbody> </table>

[!TIP] VRAM at 5.0 bpw: weights on disk + ~2 GB context overhead. Tight on 24 GB (limited context); comfortable on 32 GB+.

Download

pip install -U huggingface_hub

hf download \
  blockblockblock/Qwen3.6-35B-A3B-exl3-5.0bpw \
  --local-dir ./Qwen3.6-35B-A3B-exl3-5.0bpw

<details> <summary><b>Quantization recipe</b>  <sub>(advanced — embedded in <code>quantization_config.json</code>)</sub></summary> <br/>

Setting	Value
Format	`EXL3`
Bits per weight	`5.0`
Head bits	`8`
Calibration rows	`250`
Codebook	`MCG`
Out‑scales	`always`
Parallel mode	`enabled` (MoE expert batching)

Loaded automatically by every ExLlamaV3 loader; reproduced here for searchability.

</details>

License & use

[!IMPORTANT] Use and license follow the base model. Quantization adds no additional restrictions. Refer to the upstream repository for terms, citation, and safety documentation.

<div align="center"> <sub><i>Quantized with <a href="https://github.com/Honkware/blockquant"><b>BlockQuant</b></a>  ·  convention <code>{org}/{model}-exl3-{bpw}bpw</code></i></sub> </div>

Task: Text Generation
Library: exllamav3
Type: qwen3_5_moe
License: apache-2.0
Created: Apr 27, 2026
Updated: May 31, 2026