Downloads · 30 days
19
30% of all-time downloads
amanwalksdownthestreet/DeepSeek-V4-Flash-0731-exl3
DeepSeek-V4-Flash-0731-exl3 is a text generation model from amanwalksdownthestreet. Use it when you need the model to write or continue text. The card lists the license as mit.
Optimized exllama quantizations of deepseek-ai/DeepSeek-V4-Flash-0731 (304B total, 6 of 256 experts active per token, 1M context).
Downloads · 30 days
19
30% of all-time downloads
All-time downloads
64
Public
Repo size
979 GB
Likes
0
Public
Click a slice to open those files.
.json1.6 MB · 100%
From the Hugging Face model README
Optimized exllama quantizations of deepseek-ai/DeepSeek-V4-Flash-0731 (304B total, 6 of 256 experts active per token, 1M context).
Quantized with exllamav3 v1.4.0.
Head layer retained at full precision, attention layers across all optimized quants pinned to 4.0bpw as it is nearly free on this model — attention is 1.66% of parameters, so the boost costs ≈0.03bpw.
Given the nature of the base model, this model does not quantize well in terms of filesize/VRAM usage. The original is not BF16 — it is fp4-packed at 4.39 bits/param (166.9 GB), so there is far less headroom to reclaim than a 304B parameter count suggests, and fidelity is capped by the source. Nothing here fits in under 96 GB of VRAM. Above ≈4.4bpw the quant is larger than the original while still bounded by it, so run the original instead.
Additional quants can be generated by anyone using my measurement data in measurement.json and the base quants.
| Branch | Size |
|---|---|
| 2.39bpw-h16-opt | 85.9 GB |
| 2.95bpw-h16-opt | 105.8 GB |
| 3.33bpw-h16-opt | 119.5 GB |
| 4.02bpw-h16-opt | 143.9 GB |
2.39bpw-h16-opt is the only branch that fits a single 96 GB card.
| Branch | Size |
|---|---|
| 2.0bpw-h16 | 83.7 GB |
| 3.0bpw-h16 | 119.1 GB |
| 4.0bpw-h16 | 154.5 GB |
| 5.0bpw-h16 | 189.9 GB |
Measured with 250 calibration rows at 2048 tokens.