Downloads · 30 days
1.5K
2% of all-time downloads
gemma-2-9b-it-iMat-GGUF
qwp4w3hyb/gemma-2-9b-it-iMat-GGUF
gemma-2-9b-it-iMat-GGUF is a text generation model from qwp4w3hyb. Use it when you need the model to write or continue text. The card lists the license as gemma.
- quants done with an importance matrix for improved quantization loss - Requantized ggufs & imatrix from hf bf16 - initial version was based on f32 gguf provided by google, which had various issues - also updated for…
All-time downloads
94.7K
Public
Repo size
406 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf66.6 GB · 100%
At a glance
- Task
- Text Generation
- License
- gemma
- Access
- Public
- Created
- Jun 27, 2024
- Updated
- Jul 2, 2024
- SHA
- 9cee981a
Try a prompt
Base models
Model card
From the Hugging Face model README
Quant Infos
Updated for all recent llama.cpp fixes (final logit soft capping+sliding window+tokenizer)
- quants done with an importance matrix for improved quantization loss
- Requantized ggufs & imatrix from hf bf16
- initial version was based on f32 gguf provided by google, which had various issues
- also updated for all recent llama.cpp fixes (final logit soft capping+sliding window+tokenizer)
- Wide coverage of different gguf quant types from Q_8_0 down to IQ1_S
- experimental custom quant types
_Lwith--output-tensor-type f16 --token-embedding-type f16(same as bartowski's)
- experimental custom quant types
- Quantized with llama.cpp commit 5fac350b9cc49d0446fc291b9c4ad53666c77591 (master from 2024-07-02)
- Imatrix generated with this multi-purpose dataset by bartowski.
./imatrix -m $model_name-bf16.gguf -f calibration_datav3.txt -o $model_name.imatrix
Original Model Card
TODO
- Task
- Text Generation
- License
- gemma
- Languages
- en
- Created
- Jun 27, 2024
- Updated
- Jul 2, 2024