Downloads · 30 days
24
3% of all-time downloads
Qwen3-8B-gguf
John1604/Qwen3-8B-gguf
Qwen3-8B-gguf is a machine learning model from John1604. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Make sure you have enough ram/gpu to run. On the right of model card, you may see the size of each quantized models.
All-time downloads
798
Public
Repo size
64.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf64.4 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Oct 14, 2025
- Updated
- Oct 16, 2025
- SHA
- 831df108
Base models
Model card
From the Hugging Face model README
Qwen3-8B
Make sure you have enough ram/gpu to run. On the right of model card, you may see the size of each quantized models.
Use the model in ollama
First download and install ollama.
Command
in windows command line, or in terminal in ubuntu, type:
ollama run hf.co/John1604/Qwen3-8B-gguf:q4_k_s
(q4_k_s is the model quant type, q5_k_s, q4_k_m, ..., can also be used)
C:\Users\developer>ollama run hf.co/John1604/Qwen3-8B-gguf:q4_k_s
pulling manifest
...
verifying sha256 digest
writing manifest
success
>>>
Use the model in LM Studio
download and install LM Studio
Discover models
In the LM Studio, click "Discover" icon. "Mission Control" popup window will be displayed.
In the "Mission Control" search bar, type "John1604/Qwen3-14B-gguf" and check "GGUF", the model should be found.
Download the model.
You may choose quantized model.
Load the model.
Ask questions.
quantized models
| Type | Bits | Quality | Description |
|---|---|---|---|
| Q2_K | 2-bit | 🟥 Low | Minimal footprint; only for tests |
| Q3_K_S | 3-bit | 🟧 Low | “Small” variant (less accurate) |
| Q3_K_M | 3-bit | 🟧 Low–Med | “Medium” variant |
| Q4_K_S | 4-bit | 🟨 Med | Small, faster, slightly less quality |
| Q4_K_M | 4-bit | 🟩 Med–High | “Medium” — best 4-bit balance |
| Q5_K_S | 5-bit | 🟩 High | Slightly smaller than Q5_K_M |
| Q5_K_M | 5-bit | 🟩🟩 High | Excellent general-purpose quant |
| Q6_K | 6-bit | 🟩🟩🟩 Very High | Almost FP16 quality, larger size |
| Q8_0 | 8-bit | 🟩🟩🟩🟩 | Near-lossless baseline |
- License
- apache-2.0
- Languages
- en, zh, ja, ko
- Created
- Oct 14, 2025
- Updated
- Oct 16, 2025