Downloads · 30 days
26
3% of all-time downloads
miquliz-120b-v2.0-iMat.GGUF
dranger003/miquliz-120b-v2.0-iMat.GGUF
miquliz-120b-v2.0-iMat.GGUF is a machine learning model from dranger003. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-2.0.
GGUF importance matrix (imatrix) quants for https://huggingface.co/wolfram/miquliz-120b-v2.0 The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
All-time downloads
926
Public
Repo size
242 GB
Likes
11
Public
Repo makeup
Click a slice to open those files.
.gguf114 GB · 47%
At a glance
- License
- cc-by-nc-2.0
- Access
- Public
- Created
- Feb 15, 2024
- Updated
- Feb 18, 2024
- SHA
- 2095f4e3
Model card
From the Hugging Face model README
GGUF importance matrix (imatrix) quants for https://huggingface.co/wolfram/miquliz-120b-v2.0
The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw.
Using IQ2_XXS it seems to fit 100/141 layers using 2K context on a 24GB card.
| Layers | Context | Template |
|---|---|---|
| <pre>140</pre> | <pre>32768</pre> | <pre>[INST] {prompt} [/INST]<br>{response}</pre> |
- License
- cc-by-nc-2.0
- Created
- Feb 15, 2024
- Updated
- Feb 18, 2024