Downloads · 30 days
33
23% of all-time downloads
qwen2.5-7b-instruct-q6-k-gguf
sli3/qwen2.5-7b-instruct-q6-k-gguf
qwen2.5-7b-instruct-q6-k-gguf is a machine learning model from sli3. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is a personal GGUF quantization of Qwen/Qwen2.5-7B-Instruct, converted and quantized locally as a learning exercise in the GGUF quantization pipeline.
All-time downloads
141
Public
Repo size
6.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf6.3 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Sep 7, 2026
- Updated
- Sep 7, 2026
- SHA
- 75ec49a0
Base models
Model card
From the Hugging Face model README
Qwen2.5-7B-Instruct — Q6_K GGUF
This is a personal GGUF quantization of Qwen/Qwen2.5-7B-Instruct, converted and quantized locally as a learning exercise in the GGUF quantization pipeline.
Details
- Base model: Qwen/Qwen2.5-7B-Instruct
- Quantization: Q6_K (6-bit, K-quant)
- File size: ~6.25 GB (down from ~14.2 GB at F16)
- Tooling: llama.cpp
convert_hf_to_gguf.py→llama-quantize - Calibration: none (plain Q6_K, no imatrix)
This is not an official release and has not been extensively benchmarked against the original model — it's a straightforward Q6_K conversion using stock llama.cpp tooling, intended for personal use and experimentation.
Usage
Runs directly with llama.cpp:
llama-server -hf sli3/qwen2.5-7b-instruct-q6-k-gguf:Q6_K
or:
llama-cli -hf sli3/qwen2.5-7b-instruct-q6-k-gguf:Q6_K
License
Inherits the Apache 2.0 license of the base model.
- License
- apache-2.0
- Languages
- en
- Created
- Sep 7, 2026
- Updated
- Sep 7, 2026