Downloads · 30 days
7
3% of all-time downloads
fahmifauzi/Malaysian-Qwen2.5-7B-Instruct-Q4_K_M.gguf
Malaysian-Qwen2.5-7B-Instruct-Q4_K_M.gguf is a machine learning model from fahmifauzi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is a GGUF quantized version of mesolitica/Malaysian-Qwen2.5-7B-Instruct, converted for local inference without GPU requirements.
Downloads · 30 days
7
3% of all-time downloads
All-time downloads
210
Public
Repo size
4.7 GB
Likes
0
Public
Click a slice to open those files.
.gguf4.7 GB · 100%
From the Hugging Face model README
This is a GGUF quantized version of mesolitica/Malaysian-Qwen2.5-7B-Instruct, converted for local inference without GPU requirements.
Malaysian-Qwen2.5-7B-Instruct-Q4_K_M.gguf - Quantized model fileInstall Ollama: Download from ollama.ai
Create a Modelfile:
FROM ./Malaysian-Qwen2.5-7B-Instruct-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER top_k 40
ollama create malaysian-qwen-7b -f Modelfile
ollama run malaysian-qwen-7b
./main -m Malaysian-Qwen2.5-7B-Instruct-Q4_K_M.gguf -p "Your prompt here" -n 512
Q4_K_M offers an optimal balance between model size and quality, suitable for most use cases on consumer hardware.
This model inherits the license from the original mesolitica/Malaysian-Qwen2.5-7B-Instruct model. Please refer to the original model card for license details.
If you use this model, please cite the original model:
@misc{malaysian-qwen2.5-7b-instruct,
author = {Mesolitica},
title = {Malaysian-Qwen2.5-7B-Instruct},
year = {2024},
publisher = {HuggingFace},
url = {https://huggingface.co/mesolitica/Malaysian-Qwen2.5-7B-Instruct}
}