Downloads · 30 days
9
7% of all-time downloads
ramgpt/openbuddy-r1-0528-distill-qwen3-32b-preview0-qat-gptq-4bit
openbuddy-r1-0528-distill-qwen3-32b-preview0-qat-gptq-4bit is a machine learning model from ramgpt. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Repository: ramgpt/openbuddy-r1-0528-distill-qwen3-32b-preview0-qat-gptq-4bit
Downloads · 30 days
9
7% of all-time downloads
All-time downloads
136
Public
Parameters
32.8B
19.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors19.3 GB · 100%
How the weights are stored.
I3231.2B · 95%
From the Hugging Face model README
Repository: ramgpt/openbuddy-r1-0528-distill-qwen3-32b-preview0-qat-gptq-4bit
This is a 4-bit SGPTQ quantized version of OpenBuddy/OpenBuddy-R1-0528-Distill-Qwen3-32B-Preview0-QAT, built for efficient inference with reduced memory and compute requirements.
exllama, gptq-for-llama, vLLM with GPTQ)from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("ramgpt/openbuddy-r1-0528-distill-qwen3-32b-preview0-qat-gptq-4bit", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("ramgpt/openbuddy-r1-0528-distill-qwen3-32b-preview0-qat-gptq-4bit", device_map="auto")