Downloads · 30 days
55
33% of all-time downloads
gemma-4-12B-it-FP8-Dynamic
huseinzolkepliscicom/gemma-4-12B-it-FP8-Dynamic
gemma-4-12B-it-FP8-Dynamic is a machine learning model from huseinzolkepliscicom. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as gemma.
FP8-dynamic (compressed-tensors) quantization of google/gemma-4-12B-it, produced with llm-compressor.
All-time downloads
165
Public
Parameters
12B
13.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors13 GB · 100%
Parameter types
How the weights are stored.
F8_E4M310.9B · 91%
Base models
Model card
From the Hugging Face model README
gemma-4-12B-it — FP8 Dynamic
FP8-dynamic (compressed-tensors) quantization of google/gemma-4-12B-it, produced with
llm-compressor.
The language model is quantized to FP8 (E4M3, per-channel weights / per-token dynamic
activations). The vision + audio embedders/towers are kept in bf16 — a quantized multimodal
tower is not vLLM-servable (an FP8 gemma-4 vision tower dies with RuntimeError: Not yet supported ScalarType 46, and a quantized embed_audio/embed_vision fails vLLM weight-loading).
Verified serving on vLLM 0.23.0 (TP1), including tool-calling and reasoning:
vllm serve huseinzolkepliscicom/gemma-4-12B-it-FP8-Dynamic \
--tensor-parallel-size 1 \
--tool-call-parser gemma4 --enable-auto-tool-choice \
--reasoning-parser gemma4 \
--gpu-memory-utilization 0.9
- Type
- gemma4_unified
- License
- gemma
- Created
- Jul 15, 2026
- Updated
- Jul 15, 2026