Downloads · 30 days
58
4% of all-time downloads
qwen2.5-7b-instruct-gguf
wanhin/qwen2.5-7b-instruct-gguf
qwen2.5-7b-instruct-gguf is a machine learning model from wanhin. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
A collection of quantized Qwen2.5-7B-Instruct models in GGUF format, optimized for different hardware configurations and use cases.
All-time downloads
1.3K
Public
Repo size
28 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf28 GB · 100%
At a glance
- Access
- Public
- Created
- Jul 20, 2025
- Updated
- Jul 24, 2025
- SHA
- 69f8c2b4
Model card
From the Hugging Face model README
Qwen2.5-7B-Instruct GGUF Models
A collection of quantized Qwen2.5-7B-Instruct models in GGUF format, optimized for different hardware configurations and use cases.
🎯 Quick Start
Download Models
# Download all models
git lfs install
git clone https://huggingface.co/wanhin/qwen2.5-7b-instruct-gguf
# Or download specific models
wget https://huggingface.co/wanhin/qwen2.5-7b-instruct-gguf/resolve/main/qwen2.5-7b-instruct-q6_k.gguf
wget https://huggingface.co/wanhin/qwen2.5-7b-instruct-gguf/resolve/main/qwen2.5-7b-instruct-q4_k_m.gguf
Run Inference
# With llama.cpp
./main -m qwen2.5-7b-instruct-q6_k.gguf -n 512 --repeat_penalty 1.1
# With Python
python -c "
from llama_cpp import Llama
llm = Llama(model_path='./qwen2.5-7b-instruct-q6_k.gguf')
print(llm('Hello!', max_tokens=100)['choices'][0]['text'])
"
📦 Available Models
| Model | Size | Quality | Use Case |
|---|---|---|---|
qwen2.5-7b-instruct.gguf | 13.5 GB | Original | Best quality |
qwen2.5-7b-instruct-q8_0.gguf | 7.5 GB | Very High | High quality |
qwen2.5-7b-instruct-q4_k_m.gguf | 4.4 GB | Medium | Fast inference |
🎨 CAD Design Specialization
These models are fine-tuned for CAD design tasks and can convert natural language descriptions into structured JSON for 3D modeling operations.
📝 License
MIT License - see the model card for details.
- Created
- Jul 20, 2025
- Updated
- Jul 24, 2025