Gemma-4-31B-IT-NVFP4

gdubicki/Gemma-4-31B-IT-NVFP4

Gemma-4-31B-IT-NVFP4 is a text generation model from gdubicki. Use it when you need the model to write or continue text. It is set up for Model Optimizer. The card lists the license as apache-2.0.

Public mirror of nvidia/Gemma-4-31B-IT-NVFP4.

Downloads · 30 days

1% of all-time downloads

All-time downloads

835

Public

Parameters

20.9B

32.7 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors32.6 GB · 100%

Parameter types

How the weights are stored.

BF1610.5B · 47%

Try a prompt

Base models

nvidia/Gemma-4-31B-IT-NVFP4

Model card

From the Hugging Face model README

gdubicki/Gemma-4-31B-IT-NVFP4

Public mirror of nvidia/Gemma-4-31B-IT-NVFP4.

Weights are byte-identical to the upstream NVIDIA release. This mirror exists to provide a pinned, stable reference for the gemma4 deployment project running on DGX Spark (GB10). Use the upstream repo if you don't need a pinned copy.

Credits

Base model: google/gemma-4-31B-it by Google DeepMind
NVFP4 quantization: NVIDIA via ModelOpt
License: Apache-2.0 (Gemma license)

Verified performance

Measured on DGX Spark (GB10 Blackwell, SM12.1, 128 GB unified LPDDR5X) with vLLM 0.19.1, Marlin NVFP4 backend, BF16 KV cache:

Test	Prompt	Output	Time	Decode tok/s
EN — short Q&A	25	8	1.3s	6.1
EN — chain-of-thought	40	126	18.3s	6.9
PL — short explanation	30	116	16.9s	6.9
Python code generation	36	400	57.8s	6.9

Steady-state decode: ~6.9 tok/s (memory-bandwidth bound; Marlin runs NVFP4 as W4A16 GEMM since SM12.1 has no native CUTLASS FP4 kernel).

Usage

docker run --rm --runtime=nvidia --gpus all \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_NVFP4_GEMM_BACKEND=marlin \
  vllm/vllm-openai:gemma4-cu130 \
  gdubicki/Gemma-4-31B-IT-NVFP4 \
  --quantization modelopt_fp4 \
  --dtype bfloat16 \
  --gpu-memory-utilization 0.40 \
  --max-model-len 32768

Full deployment scripts: https://github.com/grzegorzdubicki-ai/gemma4

Task: Text Generation
Library: Model Optimizer
Type: gemma4
License: apache-2.0
Created: Apr 17, 2026
Updated: Apr 17, 2026