Downloads · 30 days
223
2% of all-time downloads
DeepSeek-V4-Flash-GGUF
tarruda/DeepSeek-V4-Flash-GGUF
DeepSeek-V4-Flash-GGUF is a text generation model from tarruda. Use it when you need the model to write or continue text. The card lists the license as mit.
GGUF quantizations for deepseek-ai/DeepSeek-V4-Flash
All-time downloads
9.1K
Public
Repo size
491 GB
Likes
21
Public
Repo makeup
Click a slice to open those files.
.gguf491 GB · 100%
At a glance
- Task
- Text Generation
- License
- mit
- Access
- Public
- Created
- Jun 30, 2026
- Updated
- Jul 2, 2026
- SHA
- 6d584186
Try a prompt
Base models
Model card
From the Hugging Face model README
DeepSeek V4 Flash GGUF
GGUF quantizations for deepseek-ai/DeepSeek-V4-Flash
DeepSeek published the original model weights in MXFP4, so the MXFP4 GGUFs in
this repo are direct conversions of those original safetensors.
Quant Recipes
| Recipe | Quant Size | Default type | Tensor-specific overrides |
|---|---|---|---|
Q3_K | 119297.23 MiB (3.52 BPW) | Q6_K | ffn_down_exps=q3_k, ffn_gate_exps=q3_k, ffn_up_exps=q3_k |
IQ3_XXS | 106912.23 MiB (3.15 BPW) | Q6_K | ffn_down_exps=iq3_xxs, ffn_gate_exps=iq3_xxs, ffn_up_exps=iq3_xxs |
Q2_K | 92465.23 MiB (2.73 BPW) | Q6_K | ffn_down_exps=q2_k, ffn_gate_exps=q2_k, ffn_up_exps=q2_k |
Usage
This is the script I use to run:
#!/bin/sh -e
model="./IQ3_XXS/DeepSeek-V4-Flash-IQ3_XXS-00001-of-00004.gguf"
ctx=131072
parallel=1
ctx_size=$((ctx * parallel))
llama-server --no-mmap --no-warmup \
--model $model --ctx-size $ctx_size -np $parallel \
--temp 1.0 --top-p 1.0
- Task
- Text Generation
- License
- mit
- Created
- Jun 30, 2026
- Updated
- Jul 2, 2026