Downloads · 30 days
22
3% of all-time downloads
Qwen3-32B-AWQ-INT4
drawais/Qwen3-32B-AWQ-INT4
Qwen3-32B-AWQ-INT4 is a text generation model from drawais. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
INT4 quantization of Qwen/Qwen3-32B. Built to run on a single 24 GB+ GPU.
All-time downloads
646
Public
Parameters
32.8B
19.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors19.3 GB · 100%
Parameter types
How the weights are stored.
I3231.2B · 95%
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3-32B-AWQ-INT4
INT4 quantization of Qwen/Qwen3-32B. Built to run on a single 24 GB+ GPU.
Footprint
| Source params | 32B |
| Quantized weights | ~18 GB on disk |
| Inference VRAM (incl. KV cache @ 32K context) | ~24 GB |
Fits any 24 GB+ GPU: RTX 3090 / 4090 / 5090, A5000, A6000, A100 40GB, etc.
Bench
Scored on drawais/needle-1M-bench-mvp (50K-token haystack, real arxiv text):
| Metric | Score |
|---|---|
| Overall recall | 100.0% |
| Paper-anchored | 100.0% |
| Synthetic codes | 100.0% |
Quick start
vllm serve drawais/Qwen3-32B-AWQ-INT4 --quantization awq_marlin --max-model-len 32768
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("drawais/Qwen3-32B-AWQ-INT4")
model = AutoModelForCausalLM.from_pretrained("drawais/Qwen3-32B-AWQ-INT4", device_map="auto")
Context length
Native: 40,960 tokens (inherits from base model). For longer contexts, enable YaRN rope-scaling per the base model's config.
License
Apache 2.0 (inherits from base model).
- Task
- Text Generation
- Type
- qwen3
- License
- apache-2.0
- Languages
- en
- Created
- Apr 28, 2026
- Updated
- Apr 28, 2026