Downloads · 30 days
24
1% of all-time downloads
Gemma-4-31B-IT-NVFP4-24GB-compact
Neural-ICE/Gemma-4-31B-IT-NVFP4-24GB-compact
Gemma-4-31B-IT-NVFP4-24GB-compact is a text generation model from Neural-ICE. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Compact NVFP4 quantized version of google/gemma-4-31B-it for vLLM.
All-time downloads
3K
Public
Parameters
16.6B
20.5 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors20.4 GB · 100%
Parameter types
How the weights are stored.
U814.6B · 79%
Try a prompt
Base models
Model card
From the Hugging Face model README
Gemma-4-31B-IT-NVFP4-24GB-compact
Compact NVFP4 quantized version of google/gemma-4-31B-it for vLLM.
Quantization Profile
- Text MLP:
NVFP4 - Text self-attention:
NVFP4 - Token embeddings:
NVFP4 lm_head: higher precision- Vision tower and vision embeddings: higher precision
- KV cache:
FP8
Usage
vllm serve Neural-ICE/Gemma-4-31B-IT-NVFP4-24GB-compact \
--quantization modelopt \
--gpu-memory-utilization 0.90
Official Gemma 4 vLLM recipe:
https://docs.vllm.ai/projects/recipes/en/latest/Google/Gemma4.html
Text Generation
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Neural-ICE/Gemma-4-31B-IT-NVFP4-24GB-compact",
"messages": [
{"role": "user", "content": "Explain quantum entanglement in simple terms."}
],
"max_tokens": 512
}'
- Task
- Text Generation
- Library
- transformers
- Type
- gemma4
- License
- apache-2.0
- Created
- Apr 7, 2026
- Updated
- Apr 8, 2026