Downloads · 30 days
46
15% of all-time downloads
DeepSeek-R1-Distill-Qwen-32B-GGUF
ReallyFloppyPenguin/DeepSeek-R1-Distill-Qwen-32B-GGUF
DeepSeek-R1-Distill-Qwen-32B-GGUF is a machine learning model from ReallyFloppyPenguin. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
This repository contains GGUF quantizations of deepseek-ai/DeepSeek-R1-Distill-Qwen-32B.
All-time downloads
302
Public
Repo size
39.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf39.3 GB · 100%
At a glance
- Library
- gguf
- License
- apache-2.0
- Access
- Public
- Created
- Jun 17, 2025
- Updated
- Jul 5, 2025
- SHA
- 3db6fa7c
Base models
Model card
From the Hugging Face model README
deepseek-ai/DeepSeek-R1-Distill-Qwen-32B - GGUF
This repository contains GGUF quantizations of deepseek-ai/DeepSeek-R1-Distill-Qwen-32B.
About GGUF
GGUF is a quantization method that allows you to run large language models on consumer hardware by reducing the precision of the model weights.
Files
| Filename | Quant type | File Size | Description |
|---|---|---|---|
| model-fp16.gguf | FP16 | Variable | fp16 quantization |
| model-q4_0.gguf | Q4_0 | Small | 4-bit quantization |
| model-q4_1.gguf | Q4_1 | Small | 4-bit quantization (higher quality) |
| model-q5_0.gguf | Q5_0 | Medium | 5-bit quantization |
| model-q5_1.gguf | Q5_1 | Medium | 5-bit quantization (higher quality) |
| model-q8_0.gguf | Q8_0 | Large | 8-bit quantization |
Usage
You can use these models with llama.cpp or any other GGUF-compatible inference engine.
llama.cpp
./llama-cli -m model-q4_0.gguf -p "Your prompt here"
Python (using llama-cpp-python)
from llama_cpp import Llama
llm = Llama(model_path="model-q4_0.gguf")
output = llm("Your prompt here", max_tokens=512)
print(output['choices'][0]['text'])
Original Model
This is a quantized version of deepseek-ai/DeepSeek-R1-Distill-Qwen-32B. Please refer to the original model card for more information about the model's capabilities, training data, and usage guidelines.
Conversion Details
- Converted using llama.cpp
- Original model downloaded from Hugging Face
- Multiple quantization levels provided for different use cases
License
This model inherits the license from the original model. Please check the original model's license for usage terms.
- Library
- gguf
- License
- apache-2.0
- Languages
- en
- Created
- Jun 17, 2025
- Updated
- Jul 5, 2025