Downloads · 30 days
4
8% of all-time downloads
Meta-Llama-3-8B-Instruct-4bit-Compressed
anoopbhargavakm87/Meta-Llama-3-8B-Instruct-4bit-Compressed
Meta-Llama-3-8B-Instruct-4bit-Compressed is a text generation model from anoopbhargavakm87. Use it when you need the model to write or continue text.
This repository contains a high-efficiency 4-bit compressed version of the model. It is optimized using bitsandbytes (NF4 precision) to run on smaller graphics cards or consumer hardware.
All-time downloads
53
Public
Parameters
8.2B
5.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.7 GB · 100%
Parameter types
How the weights are stored.
U87.2B · 87%
Try a prompt
Model card
From the Hugging Face model README
4-bit Quantized Model
This repository contains a high-efficiency 4-bit compressed version of the model. It is optimized using bitsandbytes (NF4 precision) to run on smaller graphics cards or consumer hardware.
Hardware Setup Used
Processed and generated on enterprise-grade hardware utilizing NVIDIA Multi-Instance GPU (MIG) architecture and high-capacity server systems.
Quick Start Code
To use this compressed version in your own Jupyter notebook, run:
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("YOUR_USERNAME/YOUR_REPO_NAME", device_map="auto")
- Task
- Text Generation
- Type
- llama
- Created
- Jun 23, 2026
- Updated
- Jun 23, 2026