Downloads · 30 days
50
23% of all-time downloads
Qwen3-8B-Q5_K_M
Mukilan06/Qwen3-8B-Q5_K_M
Qwen3-8B-Q5_K_M is a text generation model from Mukilan06. Use it when you need the model to write or continue text. It is set up for llama-cpp-python. The card lists the license as apache-2.0.
Qwen3-8B in Q5KM GGUF format for CPU/GPU inference with llama.cpp or llama-cpp-python.
All-time downloads
222
Public
Repo size
5.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf5.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- llama-cpp-python
- License
- apache-2.0
- Access
- Public
- Created
- Jul 25, 2026
- Updated
- Jul 25, 2026
- SHA
- ea0345f6
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3-8B-Q5_K_M
Qwen3-8B in Q5_K_M GGUF format for CPU/GPU inference with llama.cpp or llama-cpp-python.
- Source base model: Qwen/Qwen3-8B
- Original GGUF source: Qwen/Qwen3-8B-GGUF
- Quantization: Q5_K_M (~5.85 GB)
- License: Apache 2.0
Usage
py -3.11 -m pip install llama-cpp-python transformers
from llama_cpp import Llama
from transformers import AutoTokenizer
llm = Llama(model_path="Qwen3-8B-Q5_K_M.gguf", n_ctx=8192)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
messages = [{"role": "user", "content": "Hello!"}]
prompt = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
out = llm(prompt, max_tokens=256, stop=[tokenizer.eos_token])
print(out["choices"][0]["text"])
- Task
- Text Generation
- Library
- llama-cpp-python
- License
- apache-2.0
- Created
- Jul 25, 2026
- Updated
- Jul 25, 2026