Downloads · 30 days
58
5% of all-time downloads
iAkashPaul/gemma-7b-it-gguf
gemma-7b-it-gguf is a machine learning model from iAkashPaul. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
Contains Q4 & Q8 quantized GGUFs for google/gemma
Downloads · 30 days
58
5% of all-time downloads
All-time downloads
1.2K
Public
Repo size
13.9 GB
Likes
2
Public
Click a slice to open those files.
.gguf13.9 GB · 100%
From the Hugging Face model README
Contains Q4 & Q8 quantized GGUFs for google/gemma
| Variant | Device | Perf |
|---|---|---|
| Q4 | RTX 2070S | 22 tok/s |
| M1 Pro 10-core GPU | 28 tok/s | |
| Q8 | RTX 2070S | 7 tok/s (could only offload 23/29 layers to GPU) |
| M1 Pro 10-core GPU | 17 tok/s |