Downloads · 30 days
309
13% of all-time downloads
gemma-4-31B-it-fp8
vrfai/gemma-4-31B-it-fp8
gemma-4-31B-it-fp8 is a text generation model from vrfai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as gemma.
FP8 quantized version of google/gemma-4-31B-it (31B params, server model). Produced and maintained by vrfai.
All-time downloads
2.3K
Public
Parameters
31.3B
75.1 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors41.7 GB · 100%
Parameter types
How the weights are stored.
F8_E4M320.8B · 67%
Try a prompt
Base models
Model card
From the Hugging Face model README
gemma-4-31B-it-fp8
FP8 quantized version of google/gemma-4-31B-it (31B params, server model). Produced and maintained by vrfai.
Quantization Details
This model was quantized using NVIDIA ModelOpt with the following configurations:
| Property | Value |
|---|---|
| Base model | google/gemma-4-31B-it |
| Quant method | NVIDIA ModelOpt (FP8 E4M3 - num_bits: (4, 3)) |
| Weight scheme | Per-channel (axis: 0) |
| Input activation | Dynamic Per-token (type: dynamic) |
| Calibration algorithm | max |
Usage
You can deploy this model using vLLM with the modelopt quantization backend. Please ensure you refer to the vLLM documentation for Gemma 4 for advanced serving options.
vllm serve vrfai/gemma-4-31B-it-fp8
--quantization modelopt \
--max-model-len 32768 \
--max-num-seqs 128 \
--max-num-batched-tokens 8192 \
--gpu-memory-utilization 0.95 \
--kv-cache-dtype fp8 \
--enable-prefix-caching \
--enable-auto-tool-choice \
--reasoning-parser gemma4 \
--tool-call-parser gemma4 \
--async-scheduling \
--trust-remote-code
- Task
- Text Generation
- Library
- transformers
- Type
- gemma4
- License
- gemma
- Created
- Apr 17, 2026
- Updated
- Apr 24, 2026