Downloads · 30 days
11
3% of all-time downloads
ermiaazarkhalili/Qwen2.5-3B-SFT-UltraChat-GGUF
Qwen2.5-3B-SFT-UltraChat-GGUF is a machine learning model from ermiaazarkhalili. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-4.0.
GGUF quantized versions of Qwen2.5-3B-SFT-UltraChat for efficient CPU and mixed CPU/GPU inference.
Downloads · 30 days
11
3% of all-time downloads
All-time downloads
343
Public
Repo size
7.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf7.4 GB · 100%
From the Hugging Face model README
GGUF quantized versions of Qwen2.5-3B-SFT-UltraChat for efficient CPU and mixed CPU/GPU inference.
| Quantization | Size | Description |
|---|---|---|
| Q4_K_M | ~40% of original | Best balance of quality and size |
| Q5_K_M | ~50% of original | Higher quality, larger size |
| Q8_0 | ~80% of original | Highest quality quantization |
# Pull and run directly from HuggingFace
ollama pull hf.co/ermiaazarkhalili/Qwen2.5-3B-SFT-UltraChat-GGUF:Q4_K_M
ollama run hf.co/ermiaazarkhalili/Qwen2.5-3B-SFT-UltraChat-GGUF:Q4_K_M "Hello!"
# Download the GGUF file
huggingface-cli download ermiaazarkhalili/Qwen2.5-3B-SFT-UltraChat-GGUF \
qwen2.5_3b_sft_ultrachat-q4_k_m.gguf --local-dir ./models
# Run inference
./llama-cli -m ./models/qwen2.5_3b_sft_ultrachat-q4_k_m.gguf \
-p "What is the capital of France?" -n 128
This is a quantized version of Qwen2.5-3B-SFT-UltraChat, which was fine-tuned on the HuggingFaceH4/ultrachat_200k dataset using SFT.
See the source model card for full training details and usage examples.
CC-BY-NC-4.0 (same as source model)
If you use this model in your research or applications, please cite:
@misc{azarkhalili2026_qwen2_5_3b_sft_ultrachat_gguf,
author = {Azarkhalili, Behrooz},
title = {Qwen2.5-3B-SFT-UltraChat-GGUF},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/ermiaazarkhalili/Qwen2.5-3B-SFT-UltraChat-GGUF}
}
To generate a citable DOI, click "Cite this model" on the model page.
Trained using SFT (Supervised Fine-Tuning) on HuggingFaceH4/ultrachat_200k.
Base (merged) model: Qwen2.5-3B-SFT-UltraChat