Downloads · 30 days
83
12% of all-time downloads
ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF
Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF is a machine learning model from ermiaazarkhalili. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-4.0.
GGUF quantized versions of ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100 for use with llama.cpp, Ollama, LM Studio, and other GGUF-compatible tools.
Downloads · 30 days
83
12% of all-time downloads
All-time downloads
703
Public
Repo size
1.3 GB
Likes
0
Public
Click a slice to open those files.
.gguf1.3 GB · 100%
From the Hugging Face model README
GGUF quantized versions of ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100 for use with llama.cpp, Ollama, LM Studio, and other GGUF-compatible tools.
| File | Quantization | Quality | Use Case |
|---|---|---|---|
qwen2.5-0.5b-sft-openhermes-2.5-100-q4_k_m.gguf | Q4_K_M | Good | Recommended - Best balance of quality and size |
qwen2.5-0.5b-sft-openhermes-2.5-100-q5_k_m.gguf | Q5_K_M | Better | Higher quality, moderate size increase |
qwen2.5-0.5b-sft-openhermes-2.5-100-q8_0.gguf | Q8_0 | Best | Highest quality quantization |
# Download Q4_K_M (recommended)
huggingface-cli download ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF qwen2.5-0.5b-sft-openhermes-2.5-100-q4_k_m.gguf --local-dir ./models
# Download Q5_K_M (higher quality)
huggingface-cli download ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF qwen2.5-0.5b-sft-openhermes-2.5-100-q5_k_m.gguf --local-dir ./models
# Download Q8_0 (best quality)
huggingface-cli download ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF qwen2.5-0.5b-sft-openhermes-2.5-100-q8_0.gguf --local-dir ./models
# Download all quantizations
huggingface-cli download ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF --local-dir ./models
# Q4_K_M
wget https://huggingface.co/ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF/resolve/main/qwen2.5-0.5b-sft-openhermes-2.5-100-q4_k_m.gguf
# Q5_K_M
wget https://huggingface.co/ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF/resolve/main/qwen2.5-0.5b-sft-openhermes-2.5-100-q5_k_m.gguf
# Q8_0
wget https://huggingface.co/ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF/resolve/main/qwen2.5-0.5b-sft-openhermes-2.5-100-q8_0.gguf
# Pull specific quantization
ollama pull hf.co/ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF:Q4_K_M
# Or create from local file
cat > Modelfile << EOF
FROM ./qwen2.5-0.5b-sft-openhermes-2.5-100-q4_k_m.gguf
EOF
ollama create qwen2.5-0.5b-sft-openhermes-2.5-100 -f Modelfile
ollama run qwen2.5-0.5b-sft-openhermes-2.5-100
# Run with llama-cli
./llama-cli -m qwen2.5-0.5b-sft-openhermes-2.5-100-q4_k_m.gguf -p "Your prompt here" -n 256
# Run as server
./llama-server -m qwen2.5-0.5b-sft-openhermes-2.5-100-q4_k_m.gguf --host 0.0.0.0 --port 8080
from llama_cpp import Llama
llm = Llama(
model_path="qwen2.5-0.5b-sft-openhermes-2.5-100-q4_k_m.gguf",
n_ctx=2048,
n_gpu_layers=-1 # Use all GPU layers
)
output = llm(
"What is machine learning?",
max_tokens=256,
temperature=0.7,
)
print(output['choices'][0]['text'])
This is a quantized version of ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100. See the original model card for:
| Property | Value |
|---|---|
| Source Model | ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100 |
| Conversion Date | 2025-12-29 |
| Quantizations | Q4_K_M, Q5_K_M, Q8_0 |
| Converter | llama.cpp |
Same license as the original model. See ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100 for details.
Converted using the Slurm Model Trainer skill
If you use this model in your research or applications, please cite:
@misc{azarkhalili2026_qwen2_5_0_5b_sft_openhermes_2_5_100_gguf,
author = {Azarkhalili, Behrooz},
title = {Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/ermiaazarkhalili/Qwen2.5-0.5B-SFT-OpenHermes-2.5-100-GGUF}
}
To generate a citable DOI, click "Cite this model" on the model page.
Trained using SFT (Supervised Fine-Tuning) on teknium/OpenHermes-2.5.
Base (merged) model: Qwen2.5-0.5B-SFT-OpenHermes-2.5-100