Downloads · 30 days
11
0% of all-time downloads
QwQ-32B-AutoRoundGPTQ-8bit
kaitchup/QwQ-32B-AutoRoundGPTQ-8bit
QwQ-32B-AutoRoundGPTQ-8bit is a text generation model from kaitchup. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is Qwen/QwQ-32B quantized with AutoRound (symmetric quantization) and serialized with the GPTQ format in 8-bit. The model has been created, tested, and evaluated by The Kaitchup. The model is compatible with vLLM…
All-time downloads
5.2K
Public
Parameters
32.8B
35.1 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors35.1 GB · 100%
Parameter types
How the weights are stored.
I3231.2B · 95%
Try a prompt
Base models
Model card
From the Hugging Face model README
Model Details
This is Qwen/QwQ-32B quantized with AutoRound (symmetric quantization) and serialized with the GPTQ format in 8-bit. The model has been created, tested, and evaluated by The Kaitchup. The model is compatible with vLLM and Transformers.
Details on the quantization process and how to use the model here: The Kaitchup
- Developed by: The Kaitchup
- Language(s) (NLP): English
- License: Apache 2.0 license
How to Support My Work
Subscribe to The Kaitchup. This helps me a lot to continue quantizing and evaluating models for free.
- Task
- Text Generation
- Library
- transformers
- Type
- qwen2
- License
- apache-2.0
- Languages
- en
- Created
- Mar 6, 2025
- Updated
- Mar 7, 2025