Downloads · 30 days
452
3% of all-time downloads
Qwen2.5-0.5B-Instruct.GGUF
medmekk/Qwen2.5-0.5B-Instruct.GGUF
Qwen2.5-0.5B-Instruct.GGUF is a machine learning model from medmekk. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
GGUF quantized versions of Qwen/Qwen2.5-0.5B-Instruct
All-time downloads
13.5K
Public
Repo size
7.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf7.8 GB · 100%
At a glance
- Access
- Public
- Created
- Feb 1, 2025
- Updated
- Feb 1, 2025
- SHA
- 7c49df1f
Model card
From the Hugging Face model README
medmekk/Qwen2.5-0.5B-Instruct.GGUF
GGUF quantized versions of Qwen/Qwen2.5-0.5B-Instruct
Available Formats:
Q2_K: Qwen2.5-0.5B-Instruct-Q2_K.ggufQ3_K_S: Qwen2.5-0.5B-Instruct-Q3_K_S.ggufQ3_K_M: Qwen2.5-0.5B-Instruct-Q3_K_M.ggufQ3_K_L: Qwen2.5-0.5B-Instruct-Q3_K_L.ggufQ4_0: Qwen2.5-0.5B-Instruct-Q4_0.ggufQ4_K_S: Qwen2.5-0.5B-Instruct-Q4_K_S.ggufQ4_K_M: Qwen2.5-0.5B-Instruct-Q4_K_M.ggufQ5_0: Qwen2.5-0.5B-Instruct-Q5_0.ggufQ5_K_S: Qwen2.5-0.5B-Instruct-Q5_K_S.ggufQ5_K_M: Qwen2.5-0.5B-Instruct-Q5_K_M.ggufQ6_K: Qwen2.5-0.5B-Instruct-Q6_K.ggufQ8_0: Qwen2.5-0.5B-Instruct-Q8_0.ggufIQ3_M_IMAT: Qwen2.5-0.5B-Instruct-IQ3_M_imat.ggufIQ3_XXS_IMAT: Qwen2.5-0.5B-Instruct-IQ3_XXS_imat.ggufQ4_K_M_IMAT: Qwen2.5-0.5B-Instruct-Q4_K_M_imat.ggufQ4_K_S_IMAT: Qwen2.5-0.5B-Instruct-Q4_K_S_imat.ggufIQ4_NL_IMAT: Qwen2.5-0.5B-Instruct-IQ4_NL_imat.ggufIQ4_XS_IMAT: Qwen2.5-0.5B-Instruct-IQ4_XS_imat.ggufQ5_K_M_IMAT: Qwen2.5-0.5B-Instruct-Q5_K_M_imat.ggufQ5_K_S_IMAT: Qwen2.5-0.5B-Instruct-Q5_K_S_imat.gguf
Usage with llama.cpp:
# CLI:
llama-cli --hf-repo medmekk/Qwen2.5-0.5B-Instruct.GGUF --hf-file MODEL_FILE -p "Your prompt"
# Server:
llama-server --hf-repo medmekk/Qwen2.5-0.5B-Instruct.GGUF --hf-file MODEL_FILE -c 2048
- Created
- Feb 1, 2025
- Updated
- Feb 1, 2025