Downloads · 30 days
468
33% of all-time downloads
Llama-3.2-1B-Instruct-GGUF
ISTA-DASLab/Llama-3.2-1B-Instruct-GGUF
Llama-3.2-1B-Instruct-GGUF is a machine learning model from ISTA-DASLab. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf.
This repository contains advanced quantized versions of Llama 3.2 1B Instruct using GPTQ quantization and GPTQ+EvoPress optimization from the DASLab GGUF Toolkit.
All-time downloads
1.4K
Public
Repo size
8.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf8.7 GB · 100%
At a glance
- Library
- gguf
- Access
- Public
- Created
- Sep 11, 2025
- Updated
- Sep 11, 2025
- SHA
- 63250199
Base models
Model card
From the Hugging Face model README
Llama-3.2-1B-Instruct GGUF DASLab Quantization
This repository contains advanced quantized versions of Llama 3.2 1B Instruct using GPTQ quantization and GPTQ+EvoPress optimization from the DASLab GGUF Toolkit.
Models
- GPTQ Uniform: High-quality GPTQ quantization at 2-6 bit precision
- GPTQ+EvoPress: Non-uniform per-layer quantization discovered via evolutionary search
Performance
Our GPTQ-based quantization methods achieve superior quality-compression tradeoffs compared to standard quantization:
- Better perplexity at equivalent bitwidths vs. naive quantization approaches
- Error-correcting updates during calibration for improved accuracy
- Optimized configurations that allocate bits based on layer sensitivity (EvoPress)
Usage
Compatible with llama.cpp and all GGUF-supporting inference engines. No special setup required.
Full documentation, evaluation results, and toolkit source: https://github.com/IST-DASLab/gguf-toolkit
- Library
- gguf
- Created
- Sep 11, 2025
- Updated
- Sep 11, 2025