Downloads · 30 days
4
13% of all-time downloads
OLMo-2-1124-7B-Instruct-4bit-Compressed
anoopbhargavakm87/OLMo-2-1124-7B-Instruct-4bit-Compressed
OLMo-2-1124-7B-Instruct-4bit-Compressed is a machine learning model from anoopbhargavakm87. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This repository contains a high-efficiency 4-bit compressed version of the model. It is optimized using bitsandbytes (NF4 precision) to run on smaller graphics cards or consumer hardware.
All-time downloads
31
Public
Parameters
7.5B
5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5 GB · 100%
Parameter types
How the weights are stored.
U86.7B · 89%
Model card
From the Hugging Face model README
4-bit Quantized Model
This repository contains a high-efficiency 4-bit compressed version of the model. It is optimized using bitsandbytes (NF4 precision) to run on smaller graphics cards or consumer hardware.
Hardware Setup Used
Processed and generated on enterprise-grade hardware utilizing NVIDIA Multi-Instance GPU (MIG) architecture and high-capacity server systems.
Quick Start Code
To use this compressed version in your own Jupyter notebook, run:
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("YOUR_USERNAME/YOUR_REPO_NAME", device_map="auto")
- Type
- olmo2
- Created
- Jun 23, 2026
- Updated
- Jun 23, 2026