Downloads · 30 days
0
roshan-soni/qwen-3.5-0.8b-q4-k-m
qwen-3.5-0.8b-q4-k-m is a machine learning model from roshan-soni. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as apache-2.0.
This repository contains a GGUF quantized version of Qwen3.5-0.8B optimized for local inference with llama.cpp and compatible applications.
Downloads · 30 days
0
Access
Public
Updated Jun 29, 2026
Repo size
542 MB
Likes
0
Public
Click a slice to open those files.
.8b-q4-k-m542 MB · 100%
From the Hugging Face model README
This repository contains a GGUF quantized version of Qwen3.5-0.8B optimized for local inference with llama.cpp and compatible applications.
The model was converted from the original Hugging Face checkpoint to GGUF format and quantized using llama.cpp's Q4_K_M quantization scheme.
Q4_K_M provides an excellent balance between:
making it suitable for laptops, desktops, and mobile devices.
| File | Description |
|---|---|
| qwen-3.5-0.8b-q4-k-m.gguf | Q4_K_M quantized GGUF model |
./llama-cli \
-m qwen-3.5-0.8b-q4-k-m.gguf \
-p "Explain machine learning in simple terms."
This repository contains a quantized derivative of the original Qwen3.5-0.8B model.
Please refer to the original model repository for training details, benchmarks, limitations, and licensing information.
This repository only provides a GGUF quantization of the original model. All credit for model training and development belongs to the Qwen team.