Downloads · 30 days
92
3% of all-time downloads
Smaug-72B-v0.1-iMat.GGUF
dranger003/Smaug-72B-v0.1-iMat.GGUF
Smaug-72B-v0.1-iMat.GGUF is a text generation model from dranger003. Use it when you need the model to write or continue text. The card lists the license as other.
GGUF importance matrix (imatrix) quants for https://huggingface.co/abacusai/Smaug-72B-v0.1 The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
All-time downloads
2.9K
Public
Repo size
135 GB
Likes
12
Public
Repo makeup
Click a slice to open those files.
.gguf135 GB · 100%
At a glance
- Task
- Text Generation
- License
- other
- Access
- Public
- Created
- Feb 11, 2024
- Updated
- Mar 15, 2024
- SHA
- bd0e0549
Try a prompt
Model card
From the Hugging Face model README
GGUF importance matrix (imatrix) quants for https://huggingface.co/abacusai/Smaug-72B-v0.1
The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
Update 2024-03-14:
- New quant IQ1_S using latest commit
4755afd1.
Update 2024-03-02:
- New quants IQ2_S/IQ2_M, requires commit a33e6a0d or later.
- The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a general purpose imatrix calibration dataset.
- This is a different calibration dataset than the previous quants I posted so we can compare the quality
Llama-2 conversation template and system prompt set to the Qwen system prompt.
| Layers | Context | Template |
|---|---|---|
| <pre>80</pre> | <pre>32768</pre> | <pre>[INST] <<SYS>><br>{instructions}<br><</SYS>><br><br>{prompt} [/INST]<br>{response}</pre> |
- Task
- Text Generation
- License
- other
- Created
- Feb 11, 2024
- Updated
- Mar 15, 2024