Downloads · 30 days
35
6% of all-time downloads
MikeKuykendall/deepseek-moe-16b-q8-0-cpu-offload-gguf
deepseek-moe-16b-q8-0-cpu-offload-gguf is a text generation model from MikeKuykendall. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Q80 quantization of DeepSeek-MoE-16B with CPU offloading support. Highest quality, near-F16 accuracy.
Downloads · 30 days
35
6% of all-time downloads
All-time downloads
554
Public
Repo size
17.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf17.4 GB · 100%
From the Hugging Face model README
Q8_0 quantization of DeepSeek-MoE-16B with CPU offloading support. Highest quality, near-F16 accuracy.
| Configuration | VRAM | Saved | Reduction |
|---|---|---|---|
| All GPU | 17.11 GB | - | - |
| CPU Offload | 2.33 GB | 14.78 GB | 86.4% |
File Size: 17 GB (from 31 GB F16)
huggingface-cli download MikeKuykendall/deepseek-moe-16b-q8-0-cpu-offload-gguf
shimmy serve --model-dirs ./models --cpu-moe
License: Apache 2.0