Downloads · 30 days
21
4% of all-time downloads
ianwesterfield/AJ-DeepSeekR1Qwen32B-v2.1.0-GGUF
AJ-DeepSeekR1Qwen32B-v2.1.0-GGUF is a machine learning model from ianwesterfield. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
GGUF quantizations of AJ v2.1.0, a LoRA fine-tune of deepseek-ai/DeepSeek-R1-Distill-Qwen-32B for the AJ orchestrator / agent system.
Downloads · 30 days
21
4% of all-time downloads
All-time downloads
538
Public
Repo size
54.7 GB
Likes
0
Public
Click a slice to open those files.
.gguf54.7 GB · 100%
From the Hugging Face model README
GGUF quantizations of AJ v2.1.0, a LoRA fine-tune of deepseek-ai/DeepSeek-R1-Distill-Qwen-32B for the AJ orchestrator / agent system.
| File | Quant | Size | Notes |
|---|---|---|---|
AJ-DeepSeekR1Qwen32B-v2.1.0-Q4_K_M.gguf | Q4_K_M | ~19 GB | Recommended for 24 GB GPUs (RTX 4090). Good quality / speed. |
AJ-DeepSeekR1Qwen32B-v2.1.0-Q8_0.gguf | Q8_0 | ~33 GB | Near-lossless. Needs 40+ GB VRAM or CPU offload. |
deepseek-ai/DeepSeek-R1-Distill-Qwen-32B (32B, BF16)ianwesterfield/AJ-DeepSeekR1Qwen32B-v2.1.0-lora for
the raw adapter.llama-server -m AJ-DeepSeekR1Qwen32B-v2.1.0-Q4_K_M.gguf \
--host 0.0.0.0 --port 8081 \
--n-gpu-layers 65 --ctx-size 8192
DeepSeek-R1 distill uses the DeepSeek chat template (<|User|> /
<|Assistant|> / <|end▁of▁sentence|>). llama-server picks this up
from the embedded tokenizer metadata; no manual template required.