Downloads · 30 days
189
100% of all-time downloads
smoothquant-gemma2-2b-g52xlarge
chris320211/smoothquant-gemma2-2b-g52xlarge
smoothquant-gemma2-2b-g52xlarge is a text generation model from chris320211. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Quantized derivative of google/gemma-2-2b-it. Not Hub fp16. Weights are the packed/runtime artifact from job 20260921T132955Z-21a810.
All-time downloads
189
Public
Parameters
3.2B
4.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.4 GB · 99%
Parameter types
How the weights are stored.
I82B · 63%
Try a prompt
Base models
Model card
From the Hugging Face model README
SmoothQuant google/gemma-2-2b-it
Quantized derivative of google/gemma-2-2b-it.
Not Hub fp16. Weights are the packed/runtime artifact from job 20260921T132955Z-21a810.
WikiText-2 (test, 2048-token windows, 65504 tokens)
Measured on NVIDIA A10G (g5.2xlarge) against the original fp16 snapshot.
Numbers copied from jobs/20260921T132955Z-21a810/benchmark.json.
| Quantized | fp16 snapshot | |
|---|---|---|
| Perplexity | 15.128110 | 14.809264 |
| NLL loss | 2.716555 | 2.695253 |
| Tokens/s (2048 prefill, after warmup) | 6322.2 | 5413.0 |
| Peak VRAM (GB) | 8.090 | 9.973 |
ppl_ratio: 1.021530 (quality_ok=True)improved_throughput: Trueimproved_vram: True
Pull
from huggingface_hub import snapshot_download
path = snapshot_download("chris320211/smoothquant-gemma2-2b-g52xlarge")
This is not a drop-in AutoModelForCausalLM.from_pretrained checkpoint.
Reload with the included quant_agent_inference_adapter.py after the method
repo (and overlay, if any) is on QUANT_AGENT_METHOD_REPO. See
quantization_config.json.
License
Base model license (MIT for Phi-3) plus the method repository license.
Keep LICENSE / NOTICE.md from the snapshot when present.
- Task
- Text Generation
- Library
- transformers
- Type
- gemma2
- License
- mit
- Created
- Sep 21, 2026
- Updated
- Sep 21, 2026