Downloads · 30 days
39
16% of all-time downloads
thesreedath/gemma-2-2b-qa-raft-GGUF
gemma-2-2b-qa-raft-GGUF is a machine learning model from thesreedath. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
On-device (offline) quantizations of thesreedath/gemma-2-2b-qa-raft, a LoRA fine-tune of google/gemma-2-2b-it for retrieval-augmented QA. Load with any llama.cpp runtime (PocketPal AI, ChatterUI, MLC, Ollama).
Downloads · 30 days
39
16% of all-time downloads
All-time downloads
246
Public
Repo size
4.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf4.5 GB · 98%
From the Hugging Face model README
On-device (offline) quantizations of thesreedath/gemma-2-2b-qa-raft, a LoRA fine-tune of google/gemma-2-2b-it for retrieval-augmented QA. Load with any llama.cpp runtime (PocketPal AI, ChatterUI, MLC, Ollama).
| file | size | use |
|---|---|---|
| Q4_K_M | ~1.71 GB | recommended for phones |
| Q8_0 | ~2.78 GB | higher quality, more RAM |
Prompt format: prepend the retrieved context as <context id=i>...</context> blocks, then Question: ..., with the system line: You are a helpful assistant. Answer using ONLY the provided context...