Downloads · 30 days
80
20% of all-time downloads
Qwen2.5-1.5B-Instruct-Q4_K_M-GGUF
Carlosm1/Qwen2.5-1.5B-Instruct-Q4_K_M-GGUF
Qwen2.5-1.5B-Instruct-Q4_K_M-GGUF is a machine learning model from Carlosm1. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is a Q4KM quantized GGUF conversion of Qwen/Qwen2.5-1.5B-Instruct optimized for on-device inference with llama.cpp.
All-time downloads
404
Public
Repo size
1.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf1.1 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- May 15, 2026
- Updated
- May 15, 2026
- SHA
- 4c52549f
Base models
Model card
From the Hugging Face model README
Qwen2.5-1.5B-Instruct Q4_K_M GGUF
This is a Q4_K_M quantized GGUF conversion of Qwen/Qwen2.5-1.5B-Instruct optimized for on-device inference with llama.cpp.
Model Details
| Property | Value |
|---|---|
| Original Model | Qwen2.5-1.5B-Instruct |
| Parameters | 1.54 billion (1.31B non-embedding) |
| Quantization | Q4_K_M (4-bit, medium quality) |
| File Size | ~1.0 GB |
| Context Window | 32,768 tokens |
| Architecture | Qwen2 (RoPE, SwiGLU, RMSNorm) |
Intended Use
This model is optimized for:
- Mobile/Edge Deployment: Runs efficiently on most iOS devices
- llama.cpp Integration: Compatible with llama.cpp and its bindings
- On-Device AI: Private, offline inference without cloud dependencies
Capabilities
- Math & Coding: Strong mathematical and programming abilities
- Reasoning: Excellent for 1.5B class model
- Multilingual: Supports 29+ languages
- Long Context: 32K token context window
- Structured Output: Great at JSON and formatted responses
Usage with llama.cpp
./llama-cli -m Qwen2.5-1.5B-Instruct-Q4_K_M.gguf -p "Your prompt here" -n 512
License
This model inherits the Apache 2.0 license from the original Qwen2.5 model.
Attribution
- Original Model: Qwen2.5-1.5B-Instruct by Qwen Team, Alibaba Cloud
- Quantization: jc-builds
- License
- apache-2.0
- Created
- May 15, 2026
- Updated
- May 15, 2026