Downloads · 30 days
40
9% of all-time downloads
gemma-4-31B-it-FP8-Dynamic
aaronday3/gemma-4-31B-it-FP8-Dynamic
gemma-4-31B-it-FP8-Dynamic is a image-text-to-text model from aaronday3. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as gemma.
Dynamic FP8 quant of google/gemma-4-31B-it, produced with llm-compressor using the FP8DYNAMIC scheme (per-tensor FP8 weights + dynamic per-token FP8 activations, no calibration data). Format is compressed-tensors (flo…
All-time downloads
437
Public
Parameters
31.3B
33.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors33.3 GB · 100%
Parameter types
How the weights are stored.
F8_E4M329.3B · 94%
Try a prompt
Base models
Model card
From the Hugging Face model README
gemma-4-31B-it-FP8-Dynamic
Dynamic FP8 quant of google/gemma-4-31B-it,
produced with llm-compressor using the
FP8_DYNAMIC scheme (per-tensor FP8 weights + dynamic per-token FP8 activations, no calibration
data). Format is compressed-tensors (float-quantized), loadable natively by vLLM.
- ~33 GB on disk (vs ~62 GB bf16) -> fits a single 48 GB GPU. FP8 compute needs sm_89+ (Ada/Hopper): an L40S works, but an Ampere A6000 cannot run fp8 kernels.
- Only the text decoder
Linearlayers are quantized. The vision tower andlm_headare kept in bf16 (small, precision-sensitive), so multimodal + output quality are preserved. - Self-quant so the provenance is trusted, and future serves skip the 62 GB pull + on-the-fly quant.
Serve with vLLM
vllm serve aaronday3/gemma-4-31B-it-FP8-Dynamic \
--max-model-len 8192 --gpu-memory-utilization 0.90 --host 0.0.0.0 --port 8000
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- gemma4
- License
- gemma
- Created
- Jul 22, 2026
- Updated
- Jul 22, 2026