Downloads · 30 days
7
9% of all-time downloads
emreyigitozturk/llama-3.2-3b-smoothquant-gptq-hybrid
llama-3.2-3b-smoothquant-gptq-hybrid is a text generation model from emreyigitozturk. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This repository contains a quantized model artifact produced in the graduation project.
Downloads · 30 days
7
9% of all-time downloads
All-time downloads
81
Public
Parameters
3.6B
3.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3 GB · 99%
How the weights are stored.
I322.8B · 78%
From the Hugging Face model README
This repository contains a quantized model artifact produced in the graduation project.
| Metric | Original | Quantized |
|---|---|---|
| Disk size (GB) | 5.98 | 2.85 |
| Avg inference time | N/A | N/A |
| Tokens/sec | N/A | N/A |
| GPU memory | N/A | N/A |
| Perplexity | 4.3407 | 4.2447 |
Use the model with the library and runtime that match the quantization technique in this repo.