Downloads · 30 days
23
6% of all-time downloads
Qwen2.5-3B-Instruct-Q4_K_M-GGUF
TurkishCodeMan/Qwen2.5-3B-Instruct-Q4_K_M-GGUF
Qwen2.5-3B-Instruct-Q4_K_M-GGUF is a machine learning model from TurkishCodeMan. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
GGUF quantized version of Qwen2.5-3B-Instruct for mobile and edge deployment.
All-time downloads
415
Public
Repo size
1.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf1.9 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Nov 25, 2025
- Updated
- Nov 25, 2025
- SHA
- c9109217
Base models
Model card
From the Hugging Face model README
Qwen2.5-3B-Instruct-Q4_K_M-GGUF
GGUF quantized version of Qwen2.5-3B-Instruct for mobile and edge deployment.
Model Details
- Base Model: Qwen/Qwen2.5-3B-Instruct
- Quantization: Q4_K_M (4-bit quantization with K-quants)
- File Size: ~1.8 GB
- Format: GGUF
Usage
With llama.cpp
./llama-cli -m Qwen2.5-3B-Instruct-Q4_K_M.gguf -p "Hello, how are you?"
With llama.swiftui (iOS)
This model is optimized for running on iOS devices using the llama.swiftui app.
- Download the model
- Copy to app's Documents folder
- Load and chat!
Chat Template
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
Performance
| Device | Tokens/sec |
|---|---|
| iPhone 15 Pro | ~15-25 t/s |
| iPhone 14 | ~10-15 t/s |
| M1 Mac | ~30-50 t/s |
License
Apache 2.0 (following the base model license)
Credits
- Original model by Qwen Team
- Quantization and mobile optimization by TurkishCodeMan
- License
- apache-2.0
- Languages
- en, tr
- Created
- Nov 25, 2025
- Updated
- Nov 25, 2025