Downloads · 30 days
108
13% of all-time downloads
AksaraLLM/AksaraLLM-Qwen-1.5B-GGUF
AksaraLLM-Qwen-1.5B-GGUF is a text generation model from AksaraLLM. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
GGUF quantizations of AksaraLLM/AksaraLLM-Qwen-1.5B for inference with llama.cpp, Ollama, LM Studio, and other GGUF runtimes.
Downloads · 30 days
108
13% of all-time downloads
All-time downloads
812
Public
Repo size
6.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf9.3 GB · 100%
From the Hugging Face model README
GGUF quantizations of AksaraLLM/AksaraLLM-Qwen-1.5B for inference with llama.cpp, Ollama, LM Studio, and other GGUF runtimes.
| File | Quant | Size | Recommended use |
|---|---|---|---|
AksaraLLM-Qwen-1.5B.f16.gguf | F16 | 3.56 GB | lossless from safetensors |
AksaraLLM-Qwen-1.5B.q8_0.gguf | Q8_0 | 1.89 GB | near-lossless, ~2× smaller |
AksaraLLM-Qwen-1.5B.q6_k.gguf | Q6_K | 1.46 GB | high quality, ~2.5× smaller |
AksaraLLM-Qwen-1.5B.q5_k_m.gguf | Q5_K_M | 1.29 GB | good quality, ~3× smaller |
AksaraLLM-Qwen-1.5B.q4_k_m.gguf | Q4_K_M | 1.12 GB | recommended default, ~4× smaller |
| Quant | Prompt eval (32 tok) | Generation (16 tok) |
|---|---|---|
q4_k_m | 23.7 tok/s | 11.8 tok/s |
So a 1.78B model at q4_k_m runs comfortably on a CPU laptop. Larger quants (q5_k_m, q6_k, q8_0) trade a bit of speed for better quality.
huggingface-cli download AksaraLLM/AksaraLLM-Qwen-1.5B-GGUF AksaraLLM-Qwen-1.5B.q4_k_m.gguf --local-dir .
./llama-cli -m AksaraLLM-Qwen-1.5B.q4_k_m.gguf -p "Indonesia adalah" -n 64
huggingface-cli download AksaraLLM/AksaraLLM-Qwen-1.5B-GGUF AksaraLLM-Qwen-1.5B.q4_k_m.gguf Modelfile --local-dir .
ollama create aksara-aksarallm-qwen-1.5b -f Modelfile
ollama run aksara-aksarallm-qwen-1.5b "Apa ibukota Indonesia?"
See AksaraLLM/AksaraLLM-Qwen-1.5B for architecture, training data, eval results, and limitations.
convert_hf_to_gguf.py from llama.cppllama-quantize from the same buildqwen2