Downloads · 30 days
45
12% of all-time downloads
Qwen3.5-2B-GGUF
vremiks/Qwen3.5-2B-GGUF
Qwen3.5-2B-GGUF is a machine learning model from vremiks. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as apache-2.0.
Custom quantized versions of the Qwen3.5-2B model in .gguf format, optimized for efficient CPU inference via llama.cpp. Designed for lightweight deployment on systems with limited RAM (4–8 GB).
All-time downloads
388
Public
Repo size
5.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf5.1 GB · 100%
At a glance
- Library
- llama.cpp
- License
- apache-2.0
- Access
- Public
- Created
- Apr 16, 2026
- Updated
- Apr 16, 2026
- SHA
- be3c41cd
Base models
Model card
From the Hugging Face model README
📦 Qwen3.5-2B GGUF (Quantized)
Custom quantized versions of the Qwen3.5-2B model in .gguf format, optimized for efficient CPU inference via llama.cpp. Designed for lightweight deployment on systems with limited RAM (4–8 GB).
📁 Available Files
| File | Size | Quantization | Effective BPW | Purpose |
|---|---|---|---|---|
Qwen3.5-2B-Q4_K_M.gguf | ~1.2 GB | Q4_K_M | ~5.03 | ✅ Primary: Best balance of quality & speed for chat/code |
Qwen3.5-2B-fp16.gguf | ~4.1 GB | FP16 | 16.0 | 🔧 Reference format for re-quantization or GPU inference |
⚡ Quick Start
1. Download
# Download only the recommended Q4_K_M version (~1.2 GB)
hf download YOUR_USERNAME/Qwen3.5-2B-GGUF \
--include "Qwen3.5-2B-Q4_K_M.gguf" \
--local-dir ./models
- Library
- llama.cpp
- License
- apache-2.0
- Languages
- en, ru
- Created
- Apr 16, 2026
- Updated
- Apr 16, 2026