Downloads · 30 days
9
20% of all-time downloads
Mistral-7B-Instruct-v0.3-4bit-Compressed
anoopbhargavakm87/Mistral-7B-Instruct-v0.3-4bit-Compressed
Mistral-7B-Instruct-v0.3-4bit-Compressed is a machine learning model from anoopbhargavakm87. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This repository contains a high-efficiency 4-bit compressed version of the model. It is optimized using bitsandbytes (NF4 precision) to run on smaller graphics cards or consumer hardware.
All-time downloads
44
Public
Parameters
7.5B
4.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.1 GB · 100%
Parameter types
How the weights are stored.
U87.2B · 96%
Model card
From the Hugging Face model README
4-bit Quantized Model
This repository contains a high-efficiency 4-bit compressed version of the model. It is optimized using bitsandbytes (NF4 precision) to run on smaller graphics cards or consumer hardware.
Hardware Setup Used
Processed and generated on enterprise-grade hardware utilizing NVIDIA Multi-Instance GPU (MIG) architecture and high-capacity server systems.
Quick Start Code
To use this compressed version in your own Jupyter notebook, run:
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("YOUR_USERNAME/YOUR_REPO_NAME", device_map="auto")
- Type
- mistral
- Created
- Jun 23, 2026
- Updated
- Jun 23, 2026