Downloads · 30 days
28
7% of all-time downloads
lokeshe09/gemma-4-26B-A4B-it-INT4-W4A16-channelwise
gemma-4-26B-A4B-it-INT4-W4A16-channelwise is a image-text-to-text model from lokeshe09. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
4-bit weight-only quantized version of google/gemma-4-26B-A4B-it.
Downloads · 30 days
28
7% of all-time downloads
All-time downloads
403
Public
Parameters
25.8B
15 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors14.9 GB · 100%
How the weights are stored.
I3224.5B · 95%
From the Hugging Face model README
4-bit weight-only quantized version of google/gemma-4-26B-A4B-it.
| Property | Value |
|---|---|
| Scheme | W4A16 (INT4 weights, BF16 activations) |
| Strategy | Channel-wise (group_size = -1) |
| Tool | llm-compressor (git main) |
| Calibration | HuggingFaceH4/ultrachat_200k · 512 samples |
| Base model | google/gemma-4-26B-A4B-it |
Gemma 4 MoE FFN dims are 2112 and 704 — not divisible by 128. Marlin (the standard A100 W4A16 kernel) requires all dims % 128 == 0. Channel-wise (group_size=-1) is the only strategy compatible with A100 (SM80).
vllm serve lokeshe09/gemma-4-26B-A4B-it-INT4-W4A16-channelwise --max-model-len 32768 --gpu-memory-utilization 0.95