Downloads · 30 days
11
16% of all-time downloads
Llama-3.1-8B-Instruct-3bit-GuidedQuant-QTIP
jusjinuk/Llama-3.1-8B-Instruct-3bit-GuidedQuant-QTIP
Llama-3.1-8B-Instruct-3bit-GuidedQuant-QTIP is a machine learning model from jusjinuk. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.1.
- Base model: meta-llama/Llama-3.1-8B-Instruct - Quantization method: BlockLDLQ with GuidedQuant Hessian - Target bit-width: 3 - Backend kernel: QTIP kernel (HYB variant) - Calibration data: RedPajama (1024 sentences…
All-time downloads
70
Public
Parameters
2.4B
4.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.7 GB · 100%
Parameter types
How the weights are stored.
I161.3B · 55%
Base models
Model card
From the Hugging Face model README
Model Card
- Base model:
meta-llama/Llama-3.1-8B-Instruct - Quantization method: BlockLDLQ with GuidedQuant Hessian
- Target bit-width: 3
- Backend kernel: QTIP kernel (HYB variant)
- Calibration data: RedPajama (1024 sentences / 4096 tokens)
- Calibration objective: Next-token prediction
- num_groups (for GuidedQuant Hessian): 1
How to run
- Follow the instruction in https://github.com/snu-mllab/GuidedQuant and https://github.com/Cornell-RelaxML/qtip
References
- Type
- llama
- License
- llama3.1
- Created
- Jun 10, 2025
- Updated
- Jun 19, 2025