Downloads · 30 days
77
35% of all-time downloads
Llama-3.1-8B-Instruct-GPTQ-Safetensors
simaai/Llama-3.1-8B-Instruct-GPTQ-Safetensors
Llama-3.1-8B-Instruct-GPTQ-Safetensors is a text generation model from simaai. Use it when you need the model to write or continue text. The card lists the license as other.
Pre-LLiMa Hugging Face checkpoint for Sima.ai compilation.
All-time downloads
218
Public
Parameters
8B
4.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.9 GB · 100%
Parameter types
How the weights are stored.
I327.5B · 93%
Try a prompt
Base models
Model card
From the Hugging Face model README
Llama-3.1-8B-Instruct GPTQ INT4/G256 Safetensors
Pre-LLiMa Hugging Face checkpoint for Sima.ai compilation.
Quantization
- All decoder
Linearlayers andlm_head: GPTQ, symmetric INT4, group size 256, static actorder. - Calibration:
HuggingFaceH4/ultrachat_200k, 512 samples, 1024 tokens, batch size 1. - No intentional mixed-precision language layers.
Full WikiText evaluation
EleutherAI/wikitext_document_level, wikitext-2-raw-v1, full matched run (2026-07-16).
| Checkpoint | Word perplexity |
|---|---|
meta-llama/Llama-3.1-8B-Instruct | 9.328598 |
| This GPTQ checkpoint | 10.255860 |
Reproduce
python quantize.py --model-path /project/mlasw/share/huggingface/models--meta-llama--Llama-3.1-8B-Instruct --output-dir /path/to/output
Validation and limitations
Saved scales were checked for finite values. Transformers load/generation smoke test is pending before release. Quantization quality can vary by language, prompt format, domain, and context length; validate the compiled deployment on target hardware.
- Task
- Text Generation
- Type
- llama
- License
- other
- Created
- Jul 16, 2026
- Updated
- Nov 10, 2026