Downloads · 30 days
1.1K
5% of all-time downloads
Meta-Llama-3.1-70B-Instruct-iMat-GGUF
qwp4w3hyb/Meta-Llama-3.1-70B-Instruct-iMat-GGUF
Meta-Llama-3.1-70B-Instruct-iMat-GGUF is a text generation model from qwp4w3hyb. Use it when you need the model to write or continue text. The card lists the license as llama3.1.
- ~Requires latest master + Rope Scaling PR.~ Rope scaling is merged, so just a a recent master is required now. - quants done with an importance matrix for improved quantization loss - Quantized ggufs & imatrix from…
All-time downloads
25.2K
Public
Repo size
1 TB
Likes
8
Public
Repo makeup
Click a slice to open those files.
.gguf799 GB · 100%
At a glance
- Task
- Text Generation
- License
- llama3.1
- Access
- Public
- Created
- Jul 25, 2024
- Updated
- Aug 6, 2024
- SHA
- 6a776877
Try a prompt
Base models
Model card
From the Hugging Face model README
Quant Infos
Requires latest master + Rope Scaling PR.Rope scaling is merged, so just a a recent master is required now.- quants done with an importance matrix for improved quantization loss
- Quantized ggufs & imatrix from hf bf16, through bf16.
safetensors bf16 -> gguf bf16 -> quantfor optimal quant loss. - Wide coverage of different gguf quant types from Q_8_0 down to IQ1_S
- experimental custom quant types
_Lwith--output-tensor-type f16 --token-embedding-type f16, which supposedly leads to better accuracy.
- experimental custom quant types
- Imatrix generated with this multi-purpose dataset by bartowski.
./imatrix -m $model_name-bf16.gguf -f calibration_datav3.txt -o $model_name.imatrix
Original Model Card:
TODO
- Task
- Text Generation
- License
- llama3.1
- Languages
- en, de, fr, it, pt, hi
- Created
- Jul 25, 2024
- Updated
- Aug 6, 2024