Downloads · 30 days
29
9% of all-time downloads
ermiaazarkhalili/Qwen3-4B-SFT-UltraChat-GGUF
Qwen3-4B-SFT-UltraChat-GGUF is a machine learning model from ermiaazarkhalili. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-4.0.
GGUF quantized versions of ermiaazarkhalili/Qwen3-4B-SFT-UltraChat for use with llama.cpp, Ollama, LM Studio, and other GGUF-compatible tools.
Downloads · 30 days
29
9% of all-time downloads
All-time downloads
339
Public
Repo size
9.7 GB
Likes
0
Public
Click a slice to open those files.
.gguf9.7 GB · 100%
From the Hugging Face model README
GGUF quantized versions of ermiaazarkhalili/Qwen3-4B-SFT-UltraChat for use with llama.cpp, Ollama, LM Studio, and other GGUF-compatible tools.
| File | Quantization | Quality | Use Case |
|---|---|---|---|
qwen3-4b-sft-ultrachat-q4_k_m.gguf | Q4_K_M | Good | Recommended - Best balance of quality and size |
qwen3-4b-sft-ultrachat-q5_k_m.gguf | Q5_K_M | Better | Higher quality, moderate size increase |
qwen3-4b-sft-ultrachat-q8_0.gguf | Q8_0 | Best | Highest quality quantization |
# Download Q4_K_M (recommended)
huggingface-cli download ermiaazarkhalili/Qwen3-4B-SFT-UltraChat-GGUF qwen3-4b-sft-ultrachat-q4_k_m.gguf --local-dir ./models
# Download Q5_K_M (higher quality)
huggingface-cli download ermiaazarkhalili/Qwen3-4B-SFT-UltraChat-GGUF qwen3-4b-sft-ultrachat-q5_k_m.gguf --local-dir ./models
# Download Q8_0 (best quality)
huggingface-cli download ermiaazarkhalili/Qwen3-4B-SFT-UltraChat-GGUF qwen3-4b-sft-ultrachat-q8_0.gguf --local-dir ./models
# Download all quantizations
huggingface-cli download ermiaazarkhalili/Qwen3-4B-SFT-UltraChat-GGUF --local-dir ./models
# Q4_K_M
wget https://huggingface.co/ermiaazarkhalili/Qwen3-4B-SFT-UltraChat-GGUF/resolve/main/qwen3-4b-sft-ultrachat-q4_k_m.gguf
# Q5_K_M
wget https://huggingface.co/ermiaazarkhalili/Qwen3-4B-SFT-UltraChat-GGUF/resolve/main/qwen3-4b-sft-ultrachat-q5_k_m.gguf
# Q8_0
wget https://huggingface.co/ermiaazarkhalili/Qwen3-4B-SFT-UltraChat-GGUF/resolve/main/qwen3-4b-sft-ultrachat-q8_0.gguf
# Pull specific quantization
ollama pull hf.co/ermiaazarkhalili/Qwen3-4B-SFT-UltraChat-GGUF:Q4_K_M
# Or create from local file
cat > Modelfile << EOF
FROM ./qwen3-4b-sft-ultrachat-q4_k_m.gguf
EOF
ollama create qwen3-4b-sft-ultrachat -f Modelfile
ollama run qwen3-4b-sft-ultrachat
# Run with llama-cli
./llama-cli -m qwen3-4b-sft-ultrachat-q4_k_m.gguf -p "Your prompt here" -n 256
# Run as server
./llama-server -m qwen3-4b-sft-ultrachat-q4_k_m.gguf --host 0.0.0.0 --port 8080
from llama_cpp import Llama
llm = Llama(
model_path="qwen3-4b-sft-ultrachat-q4_k_m.gguf",
n_ctx=2048,
n_gpu_layers=-1 # Use all GPU layers
)
output = llm(
"What is machine learning?",
max_tokens=256,
temperature=0.7,
)
print(output['choices'][0]['text'])
This is a quantized version of ermiaazarkhalili/Qwen3-4B-SFT-UltraChat. See the original model card for:
| Property | Value |
|---|---|
| Source Model | ermiaazarkhalili/Qwen3-4B-SFT-UltraChat |
| Conversion Date | 2025-12-26 |
| Quantizations | Q4_K_M, Q5_K_M, Q8_0 |
| Converter | llama.cpp |
Same license as the original model. See ermiaazarkhalili/Qwen3-4B-SFT-UltraChat for details.
Converted using the Slurm Model Trainer skill
If you use this model in your research or applications, please cite:
@misc{azarkhalili2026_qwen3_4b_sft_ultrachat_gguf,
author = {Azarkhalili, Behrooz},
title = {Qwen3-4B-SFT-UltraChat-GGUF},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/ermiaazarkhalili/Qwen3-4B-SFT-UltraChat-GGUF}
}
To generate a citable DOI, click "Cite this model" on the model page.
Trained using SFT (Supervised Fine-Tuning) on HuggingFaceH4/ultrachat_200k.
Base (merged) model: Qwen3-4B-SFT-UltraChat