Downloads · 30 days
259
6% of all-time downloads
Llama3-ChatQA-1.5-8B-iMat-GGUF
InferenceIllusionist/Llama3-ChatQA-1.5-8B-iMat-GGUF
Llama3-ChatQA-1.5-8B-iMat-GGUF is a machine learning model from InferenceIllusionist. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.
<img src="https://i.imgur.com/P68dXux.png" width="400"/
All-time downloads
4.3K
Public
Repo size
37.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf37.7 GB · 100%
At a glance
- License
- llama3
- Access
- Public
- Created
- May 21, 2024
- Updated
- May 23, 2024
- SHA
- d01ed114
Model card
From the Hugging Face model README
<img src="https://i.imgur.com/P68dXux.png" width="400"/>
Llama3-ChatQA-1.5-8B-iMat-GGUF
Quantized from fp16.
- Weighted quantizations were creating using fp16 GGUF and groups_merged-enhancedV2-TurboMini.txt in 234 chunks and n_ctx=512
- This method of calculating the importance matrix showed improvements in some areas for Mistral 7b and Llama3 8b models, see above post for details
- The enhancedv2-turbomini file appends snippets from turboderp's calibration data to the standard groups_merged.txt file
For a brief rundown of iMatrix quant performance please see this PR
<i>All quants are verified working prior to uploading to repo for your safety and convenience. </i>
<b>Tip:</b> Pick a file size under your GPU's VRAM while still allowing some room for context for best speed. You may need to pad this further depending on if you are running image gen or TTS as well.
BFloat16 model card can be found here
- License
- llama3
- Created
- May 21, 2024
- Updated
- May 23, 2024