Downloads · 30 days
39
10% of all-time downloads
deepseek-moe-16b-q2-k-cpu-offload-gguf
MikeKuykendall/deepseek-moe-16b-q2-k-cpu-offload-gguf
deepseek-moe-16b-q2-k-cpu-offload-gguf is a text generation model from MikeKuykendall. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Q2K quantization of DeepSeek-MoE-16B with CPU offloading support. Smallest size, maximum VRAM savings.
All-time downloads
401
Public
Repo size
6.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf6.7 GB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- Oct 9, 2025
- Updated
- Oct 9, 2025
- SHA
- b1f1237d
Try a prompt
Base models
Model card
From the Hugging Face model README
DeepSeek-MoE-16B Q2_K with CPU Offloading
Q2_K quantization of DeepSeek-MoE-16B with CPU offloading support. Smallest size, maximum VRAM savings.
Performance
| Configuration | VRAM | Saved | Reduction |
|---|---|---|---|
| All GPU | 7.28 GB | - | - |
| CPU Offload | 1.60 GB | 5.68 GB | 78.0% |
File Size: 6.3 GB (from 31 GB F16)
Usage
huggingface-cli download MikeKuykendall/deepseek-moe-16b-q2-k-cpu-offload-gguf
shimmy serve --model-dirs ./models --cpu-moe
License: Apache 2.0
- Task
- Text Generation
- License
- apache-2.0
- Languages
- en, zh
- Created
- Oct 9, 2025
- Updated
- Oct 9, 2025