Downloads · 30 days
30
8% of all-time downloads
Stonehallow/Qwen3.6-35B-A3B-EXL3-4.5bpw
Qwen3.6-35B-A3B-EXL3-4.5bpw is a text generation model from Stonehallow. Use it when you need the model to write or continue text. It is set up for exllamav3. The card lists the license as apache-2.0.
This repository contains a 4.5 bits-per-weight (bpw) quantization of the Qwen3.6-35B-A3B model. The conversion was performed using the ExLlamaV3 format via the convert.py script.
Downloads · 30 days
30
8% of all-time downloads
All-time downloads
398
Public
Repo size
21.5 GB
Likes
0
Public
Click a slice to open those files.
.safetensors21.4 GB · 100%
From the Hugging Face model README
This repository contains a 4.5 bits-per-weight (bpw) quantization of the Qwen3.6-35B-A3B model. The conversion was performed using the ExLlamaV3 format via the convert.py script.
Qwen3.6-35B-A3B is a sparse Mixture-of-Experts (MoE) large language model. It utilizes a fine-grained expert architecture designed for high parameter utilization and task-specific activation.
The model was quantized with the following parameters using ExLlamaV3 v0.0.29.
-hb 6) applied specifically to the output layer.This model requires an inference engine compatible with the ExLlamaV3 format, such as the official ExLlamaV3 library or backends that utilize it (e.g., TabbyAPI).
*.safetensors: Quantized weights.config.json & Metadata: Standard configuration files derived from the BF16 source to ensure architectural compatibility.