Downloads · 30 days
40
18% of all-time downloads
Llama-3.1-8B-Instruct-BioQA-AWQ
anthonyyazdaniml/Llama-3.1-8B-Instruct-BioQA-AWQ
Llama-3.1-8B-Instruct-BioQA-AWQ is a text generation model from anthonyyazdaniml. Use it when you need the model to write or continue text.
A quantized version of Llama-3.1-8B-Instruct. The model was quantized using AutoAWQ with biomedical question-answering (QA) data as calibration.
All-time downloads
217
Public
Parameters
8B
5.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.7 GB · 100%
Parameter types
How the weights are stored.
I327B · 87%
Try a prompt
Base models
Model card
From the Hugging Face model README
Llama-3.1-8B-Instruct-BioQA-AWQ
A quantized version of Llama-3.1-8B-Instruct. The model was quantized using AutoAWQ with biomedical question-answering (QA) data as calibration.
Key Details
- Base Model: Llama-3.1-8B-Instruct
- Calibration Data: Biomedical question-answering (QA)
- Template: Official Llama chat format
Quantization Config
quant_config = {
"zero_point": True,
"q_group_size": 128,
"w_bit": 4,
"version": "GEMM"
}
License
The model follows the license of the base Llama-3.1-8B-Instruct model.
- Task
- Text Generation
- Type
- llama
- Created
- Jan 13, 2025
- Updated
- Jan 13, 2025