Downloads · 30 days
9
14% of all-time downloads
Angelic_Eclipse_12B-AWQ-4bit
isola-tropicale/Angelic_Eclipse_12B-AWQ-4bit
Angelic_Eclipse_12B-AWQ-4bit is a machine learning model from isola-tropicale. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
AWQ 4bit quantization of SicariusSicarii's AngelicEclipse12B
All-time downloads
66
Public
Parameters
2.8B
8.4 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors8.4 GB · 100%
Parameter types
How the weights are stored.
I3211B · 89%
Base models
Model card
From the Hugging Face model README
AWQ 4bit quantization of SicariusSicarii's Angelic_Eclipse_12B
Quantized on a single Nvidia RTX 4090.
Recipe:
from transformers import AutoModelForCausalLM, AutoTokenizer
from llmcompressor import oneshot
from llmcompressor.modifiers.awq import AWQModifier
dataset = "gsm8k"
model_id = "/path/to/model/"
SAVE_DIR = "/save/dir/"
MAX_SEQUENCE_LENGTH = 2048
NUM_CALIBRATION_SAMPLES = 64
tokenizer = AutoTokenizer.from_pretrained(
model_id,
)
recipe = [
AWQModifier(
targets=["Linear"],
scheme="W4A16_ASYM",
ignore=["lm_head"],
)
]
oneshot(
model=model_id,
dataset=dataset,
dataset_config_name="main",
recipe=recipe,
output_dir=SAVE_DIR,
max_seq_length=MAX_SEQUENCE_LENGTH,
num_calibration_samples=NUM_CALIBRATION_SAMPLES,
)
- Type
- mistral
- License
- apache-2.0
- Created
- Dec 29, 2025
- Updated
- Dec 29, 2025