Downloads · 30 days
357
10% of all-time downloads
ThijsL202/CraneAILabs_swahili-gemma-1b-im-GGUF
CraneAILabs_swahili-gemma-1b-im-GGUF is a text generation model from ThijsL202. Use it when you need the model to write or continue text. It is set up for transformers.
- Base Model: CraneAILabs/swahili-gemma-1b - Quantization: Importance Matrix - Total Size: 6.25 GB (9 files)
Downloads · 30 days
357
10% of all-time downloads
All-time downloads
3.5K
Public
Repo size
6.7 GB
Likes
0
Public
Click a slice to open those files.
.gguf6.7 GB · 100%
From the Hugging Face model README
| Quantization | Size | Download |
|---|---|---|
| IQ2_XXS | 651MB | ⬇️ |
| Q2_K_S | 671MB | ⬇️ |
| Q2_K | 690MB | ⬇️ |
| IQ3_M | 697MB | ⬇️ |
| IQ4_XS | 714MB | ⬇️ |
| IQ4_NL | 722MB | ⬇️ |
| Q3_K_L | 752MB | ⬇️ |
| Q4_K_M | 806MB | ⬇️ |
| Q6_K | 1.0GB | ⬇️ |
./llama-cli -m CraneAILabs_swahili-gemma-1b.im-Q2_K_S.gguf -p "Your prompt here" -n 512
from llama_cpp import Llama
llm = Llama(model_path="./CraneAILabs_swahili-gemma-1b.im-Q2_K_S.gguf", n_ctx=2048)
output = llm("Your prompt here", max_tokens=512)
print(output["choices"][0]["text"])
Original model: CraneAILabs/swahili-gemma-1b
Quantized using llama.cpp