Downloads · 30 days
381
4% of all-time downloads
gemma-7b-GGUF-quantized
sayhan/gemma-7b-GGUF-quantized
gemma-7b-GGUF-quantized is a text generation model from sayhan. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
- Model creator: Google - Original model: gemma-7b-it - Terms of use
All-time downloads
10.4K
Public
Repo size
162 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf81 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Access
- Public
- Created
- Feb 21, 2024
- Updated
- Feb 23, 2024
- SHA
- ceab9417
Try a prompt
Base models
Model card
From the Hugging Face model README
Google Gemma 7B
- Model creator: Google
- Original model: gemma-7b-it
- Terms of use
Description
This repo contains GGUF format model files for Google's Gemma 7B
Original model
- Developed by: Google
Description
Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. They are text-to-text, decoder-only large language models, available in English, with open weights, pre-trained variants, and instruction-tuned variants. Gemma models are well-suited for a variety of text generation tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as a laptop, desktop or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone.
Quantizon types
| quantization method | bits | size | description | recommended |
|---|---|---|---|---|
| Q2_K | 2 | 3.09 | very small, very high quality loss | ❌ |
| Q3_K_S | 3 | 3.68 GB | very small, high quality loss | ❌ |
| Q3_K_L | 3 | 4.4 GB | small, substantial quality loss | ❌ |
| Q4_0 | 4 | 4.81 GB | legacy; small, very high quality loss | ❌ |
| Q4_K_S | 4 | 4.84 GB | medium, balanced quality | ✅ |
| Q4_K_M | 4 | 5.13 GB | medium, balanced quality | ✅ |
| Q5_0 | 5 | 5.88 GB | legacy; medium, balanced quality | ❌ |
| Q5_K_S | 5 | 5.88 GB | large, low quality loss | ✅ |
| Q5_K_M | 5 | 6.04 GB | large, very low quality loss | ✅ |
| Q6_K | 6 | 7.01 GB | very large, extremely low quality loss | ❌ |
| Q8_0 | 8 | 9.08 GB | very large, extremely low quality loss | ❌ |
| FP16 | 16 | 17.1 GB | enormous, negligible quality loss | ❌ |
Usage
You can use this model with the latest builds of LM Studio and llama.cpp.
If you're new to the world of large language models, I recommend starting with LM Studio.
- Task
- Text Generation
- Library
- transformers
- License
- other
- Languages
- en
- Created
- Feb 21, 2024
- Updated
- Feb 23, 2024