Downloads · 30 days
14
4% of all-time downloads
fahmifauzi/Malaysian-Qwen2.5-3B-Instruct-Q4_K_M-CPU-optimized.gguf
Malaysian-Qwen2.5-3B-Instruct-Q4_K_M-CPU-optimized.gguf is a machine learning model from fahmifauzi. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a GGUF quantized version of mesolitica/Malaysian-Qwen2.5-3B-Instruct, converted for local inference without GPU requirements.
Downloads · 30 days
14
4% of all-time downloads
All-time downloads
366
Public
Repo size
2.1 GB
Likes
1
Public
Click a slice to open those files.
.gguf2.1 GB · 100%
From the Hugging Face model README
This is a GGUF quantized version of mesolitica/Malaysian-Qwen2.5-3B-Instruct, converted for local inference without GPU requirements.
Malaysian-Qwen2.5-3B-Instruct-Q4_K_M.gguf - Quantized model fileInstall Ollama: Download from ollama.ai
Create a Modelfile:
FROM ./Malaysian-Qwen2.5-3B-Instruct-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER top_k 40
ollama create malaysian-qwen -f Modelfile
ollama run malaysian-qwen
./main -m Malaysian-Qwen2.5-3B-Instruct-Q4_K_M.gguf -p "Your prompt here" -n 512
Q4_K_M offers an optimal balance between model size and quality, suitable for most use cases on consumer hardware.
This model inherits the license from the original mesolitica/Malaysian-Qwen2.5-3B-Instruct model. Please refer to the original model card for license details.
If you use this model, please cite the original model:
@misc{malaysian-qwen2.5-3b-instruct,
author = {Mesolitica},
title = {Malaysian-Qwen2.5-3B-Instruct},
year = {2024},
publisher = {HuggingFace},
url = {https://huggingface.co/mesolitica/Malaysian-Qwen2.5-3B-Instruct}
}