Downloads · 30 days
22
24% of all-time downloads
Llama-3.1-8B-Instruct-bf16-quantized.w8a8
DESUCLUB/Llama-3.1-8B-Instruct-bf16-quantized.w8a8
Llama-3.1-8B-Instruct-bf16-quantized.w8a8 is a text generation model from DESUCLUB. Use it when you need the model to write or continue text. It is set up for transformers.
This is a custom W8A16 quantized version of meta-llama/Llama-3.1-8B-Instruct.
All-time downloads
92
Public
Parameters
8B
9.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors9.1 GB · 100%
Parameter types
How the weights are stored.
I87B · 87%
Try a prompt
Base models
Model card
From the Hugging Face model README
DESUCLUB/Llama-3.1-8B-Instruct-bf16-quantized.w8a8
This is a custom W8A16 quantized version of meta-llama/Llama-3.1-8B-Instruct.
Quantization Details
- Method: Custom W8A16 (8-bit weights, 16-bit activations)
- Weight precision: INT8
- Scale precision: BF16
- Quantization: Symmetric per-channel
- Zero points: None (symmetric)
Model Structure
The quantized model contains:
.weight: INT8 quantized weights.weight_scale: BF16 scale parameters (trainable)- Standard embedding and normalization layers in original precision
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
# Note: This requires custom quantization code to load properly
model = AutoModelForCausalLM.from_pretrained("DESUCLUB/Llama-3.1-8B-Instruct-bf16-quantized.w8a8")
tokenizer = AutoTokenizer.from_pretrained("DESUCLUB/Llama-3.1-8B-Instruct-bf16-quantized.w8a8")
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Jun 4, 2025
- Updated
- Jun 4, 2025