Downloads · 30 days
195
5% of all-time downloads
DeepSeek-R1-Distill-Qwen-32B-gguf
John1604/DeepSeek-R1-Distill-Qwen-32B-gguf
DeepSeek-R1-Distill-Qwen-32B-gguf is a machine learning model from John1604. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is distilled Deepseek-r1. Make sure you have enough ram/gpu to run. On the right of model card, you may see the size of each quantized models.
All-time downloads
4K
Public
Repo size
254 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.gguf254 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Oct 12, 2025
- Updated
- Oct 12, 2025
- SHA
- c8332a77
Base models
Model card
From the Hugging Face model README
Deepseek-r1 32B
This is distilled Deepseek-r1. Make sure you have enough ram/gpu to run. On the right of model card, you may see the size of each quantized models.
Use the model in ollama
First download and install ollama.
Command
in windows command line, or in terminal in ubuntu, type:
ollama run hf.co/John1604/DeepSeek-R1-Distill-Qwen-32B-gguf:q3_k_m
(q3_k_m is the model quant type, q5_k_s, q4_k_m, ..., can also be used)
C:\Users\developer>ollama run hf.co/John1604/DeepSeek-R1-Distill-Qwen-32B-gguf:q3_k_m
pulling manifest
...
verifying sha256 digest
writing manifest
success
>>> Send a message (/? for help)
Use the model in LM Studio
download and install LM Studio
Discover models
In the LM Studio, click "Discover" icon. "Mission Control" popup window will be displayed.
In the "Mission Control" search bar, type "John1604/DeepSeek-R1-Distill-Qwen-32B-gguf" and check "GGUF", the model should be found.
Download the model.
you may choose the quantized type in the download window.
Load the model.
Load the model to the LM Studio.
Ask questions.
quantized models
| Type | Bits | Quality | Description |
|---|---|---|---|
| Q2_K | 2-bit | 🟥 Low | Minimal footprint; only for tests |
| Q3_K_S | 3-bit | 🟧 Low | “Small” variant (less accurate) |
| Q3_K_M | 3-bit | 🟧 Low–Med | “Medium” variant |
| Q4_K_S | 4-bit | 🟨 Med | Small, faster, slightly less quality |
| Q4_K_M | 4-bit | 🟩 Med–High | “Medium” — best 4-bit balance |
| Q5_K_S | 5-bit | 🟩 High | Slightly smaller than Q5_K_M |
| Q5_K_M | 5-bit | 🟩🟩 High | Excellent general-purpose quant |
| Q6_K | 6-bit | 🟩🟩🟩 Very High | Almost FP16 quality, larger size |
| Q8_0 | 8-bit | 🟩🟩🟩🟩 | Near-lossless baseline |
- License
- apache-2.0
- Languages
- en, zh
- Created
- Oct 12, 2025
- Updated
- Oct 12, 2025