Downloads · 30 days
21
29% of all-time downloads
gemma-4-E2B-it-NVFP4KV-calib
jethachan/gemma-4-E2B-it-NVFP4KV-calib
gemma-4-E2B-it-NVFP4KV-calib is a machine learning model from jethachan. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as gemma.
google/gemma-4-E2B-it (weights unchanged, bf16) plus per-layer kscale/vscale for a 4-bit float KV cache, produced with llm-compressor 0.12 / compressed-tensors 0.17.1 (kvcachescheme: {numbits: 4, type: float, strategy…
All-time downloads
73
Public
Parameters
5.5B
11 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors11 GB · 100%
At a glance
- License
- gemma
- Model type
- gemma4
- Access
- Public
- Created
- Sep 6, 2026
- Updated
- Sep 6, 2026
- SHA
- a58ce62f
Base models
Model card
From the Hugging Face model README
Gemma-4-E2B-it with calibrated NVFP4 KV-cache scales
google/gemma-4-E2B-it (weights unchanged, bf16) plus per-layer k_scale/v_scale for a 4-bit float KV cache,
produced with llm-compressor 0.12 / compressed-tensors 0.17.1 (kv_cache_scheme: {num_bits: 4, type: float, strategy: tensor, observer: memoryless_minmax},
256 HuggingFaceH4/ultrachat_200k samples at seq 2048; recipe in recipe.yaml).
Purpose: a checkpoint that vLLM can serve with --kv-cache-dtype nvfp4 on consumer Blackwell (RTX 5090 / GB10) via the FlashInfer FA2 path
of vllm-project/vllm#46329, and the model behind vLLM's tests/v1/e2e/general/test_gemma4_nvfp4_kv_sm12x.py. Public "NVFP4" Gemma-4
checkpoints carry no KV scales and produce garbage with an NVFP4 KV cache; this one does not.
vllm serve jethachan/gemma-4-E2B-it-NVFP4KV-calib --kv-cache-dtype nvfp4 --language-model-only
Repro, probes and results: https://github.com/jethac/vllm-gemma4-nvfp4-kv-repro
- Type
- gemma4
- License
- gemma
- Created
- Sep 6, 2026
- Updated
- Sep 6, 2026