Downloads · 30 days
35
22% of all-time downloads
DeepSeek-V3-0324-GGUF
Doctor-Shotgun/DeepSeek-V3-0324-GGUF
DeepSeek-V3-0324-GGUF is a machine learning model from Doctor-Shotgun. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a custom quant of deepseek-ai/DeepSeek-V3-0324 that has the following: - Q80 for the default quantization type (attention, shared experts, etc.) - Q4K for the FFNUP and FFNGATE tensors - Q5K for the FFNDOWN te…
All-time downloads
159
Public
Repo size
414 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf414 GB · 100%
At a glance
- Access
- Public
- Created
- Dec 19, 2025
- Updated
- Dec 20, 2025
- SHA
- 09ca1bd8
Base models
Model card
From the Hugging Face model README
This is a custom quant of deepseek-ai/DeepSeek-V3-0324 that has the following:
- Q8_0 for the default quantization type (attention, shared experts, etc.)
- Q4_K for the FFN_UP and FFN_GATE tensors
- Q5_K for the FFN_DOWN tensors
The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization.
This is derived from the quant provided by AesSedai with additional keys added for expert group selection.
Model is additionally split with --no-tensor-first-split to enable easier editing of metadata.
- Created
- Dec 19, 2025
- Updated
- Dec 20, 2025